How can you install private Python packages in requirements using serverless-deploy:1.2.0?

[Sam Mahr]

I have a private package I'd like to install and package with an AWS Lambda deployment using serverless deploy pipe. 

I have the following package format in the `requirements.txt` file

git+ssh://git@bitbucket.org/orgname/thelibrary.git@v0.1.1

This works outside of the serverless deploy pipe when running `pip install -r requirements.txt`.

I've set up an SSH key on the repository for the serverless application and I've added the SSH key to the private package repository.

When I am trying to install within the serverless deploy pipe, I get this error ( also scrubbed org name and repository name in the output):

Generated requirements from /opt/atlassian/pipelines/agent/build/requirements.txt in /opt/atlassian/pipelines/agent/build/.serverless/requirements.txt
Installing requirements from "/root/.cache/serverless-python-requirements/0c5359442d26dbfb99408628d71fd97eec048649b5f2c75197c9f0eedeb2270e_x86_64_slspyc/requirements.txt"
Using download cache directory /root/.cache/serverless-python-requirements/downloadCacheslspyc
Running ...
Stdout: Collecting git+ssh://****@bitbucket.org/orgname/thelibrary.git@v0.1.1 (from -r /root/.cache/serverless-python-requirements/0c5359442d26dbfb99408628d71fd97eec048649b5f2c75197c9f0eedeb2270e_x86_64_slspyc/requirements.txt (line 1))
Cloning ssh://****@bitbucket.org/orgname/thelibrary.git (to revision v0.1.1) to /tmp/pip-req-build-_p1kws94
Stderr: Running command git clone --filter=blob:none --quiet 'ssh://****@bitbucket.org/orgname/thelibrary.git' /tmp/pip-req-build-_p1kws94
Host key verification failed.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
error: subprocess-exited-with-error

× git clone --filter=blob:none --quiet 'ssh://****@bitbucket.org/orgname/thelibrary.git' /tmp/pip-req-build-_p1kws94 did not run successfully.
│ exit code: 128
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error
× git clone --filter=blob:none --quiet 'ssh://****@bitbucket.org/orgname/thelibrary.git' /tmp/pip-req-build-_p1kws94 did not run successfully.
│ exit code: 128
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
WARNING: You are using pip version 22.0.4; however, version 23.2.1 is available.
You should consider upgrading via the '/usr/local/bin/python3.9 -m pip install --upgrade pip' command.

Is it possible to install private packages in this method within the serverless deploy pipe or will I have to create my own script to deploy serverless applications without using the pipe?

[Oleksandr Kyrdan · Accepted solution]

Hi @Sam Mahr

Thank you for your question!

According to your case, we suggest you to try PRE_EXECUTION_SCRIPT variable to execute script file with some logic with your SSH_KEY.

Please, follow the example how to setup SSH inside a pipe's docker container.

Example passing pre-execution hook script file (file should have at least read and execute permissions):

script:
  - chmod 005 setup-ssh-script.sh
  - pipe: atlassian/serverless-deploy:1.3.0
    variables:
      AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID
      AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY
      PRE_EXECUTION_SCRIPT: '.setup-ssh-script.sh'

Best regards,
Oleksandr Kyrdan

[Sam Mahr]

Thanks! I was already using a pre-execution script. Added the ssh dir setup and it worked smoothly