Create
cancel
Showing results for 
Search instead for 
Did you mean: 
Sign up Log in

It's not the same without you

Join the community to find out what other Atlassian users are discussing, debating and creating.

Atlassian Community Hero Image Collage

Where does pip install packages in pipelines?

Edited

Hi,

I have just started using pipelines, and I have a yml file as follows:

 

image: python:3.6.4

clone
:
depth: full

pipelines
:
default:
-
step: name: Installing dependancies
caches:
-
pip
script:
-
pip install -r requirements.txt -vvv
-
step:
name: Debugging dependancy installs and locations
script:
-
ls -altr /usr/local/lib/python3.6/site-packages
-
ls -altr /usr/local/bin
-
which pip
-
which python
-
python -c "import site; print(site.getsitepackages())"
-
step:
name: Installing local app script:
-
python setup.py install
-
ls -altr /usr/local/lib/python3.6/site-packages
-
step:
name: Setting environment variables
script:
-
export FLASK_APP=stackademicdata
-
step:
name: Run Flask app
caches:
-
flask
script:
-
python -m flask run
-
step:
deployment: test
name: Deploy to Heroku
script:
-
git push https://heroku:$HEROKU_API_KEY@git.heroku.com/$HEROKU_DATA_APP_NAME.git HEADdefinitions:
caches:
flask: /usr/local/bin/flask

 

the contents of my requirement.txt is:

 

flask
flask-sqlalchemy
click
gunicorn
pytest
setuptools
jsonify
requests
entrypoints
decorators
psycopg2
sqlalchemy
jsonschema
flask-wtf

 

the output states that all packages in this step have been installed correctly:

 

Successfully installed 

Jinja2-2.10
MarkupSafe-1.0
WTForms-2.1
Werkzeug-0.14.1
attrs-17.4.0
certifi-2018.1.18
chardet-3.0.4
click-6.7
decorators-0.1.1
entrypoints-0.2.3
flask-0.12.2 flask-sqlalchemy-2.3.2
flask-wtf-0.14.2
gunicorn-19.7.1
idna-2.6
itsdangerous-0.24
jsonify-0.5
jsonschema-2.6.0
pluggy-0.6.0
psycopg2-2.7.4
py-1.5.2
pytest-3.4.2
requests-2.18.4
six-1.11.0
sqlalchemy-1.2.5
urllib3-1.22

Cleaning up..

 

 

Therefore I would expect to be able to use those packages in the standard way, with CLI control. For example, with the flask package, in my build file I need to be able to run 'flask run' to test whether the application starts correctly.

 

However, the build always fails on that step, with the error:

+ python -m flask run/usr/local/bin/python: No module named flask

 

I have included debugging steps, but for the life of me I cant figure out where pip installs the packages?!

 

This is the output of the debugging step commands:

 

+ ls -altr /usr/local/lib/python3.6/site-packages
total 96
-rw-r--r--. 1 root staff 119 Mar 14 17:49 README.txt
drwxr-sr-x. 1 root staff 4096 Mar 14 17:49 ..
drwxr-sr-x. 2 root staff 4096 Mar 14 17:49 wheel-0.30.0.dist-info
drwxr-sr-x. 5 root staff 4096 Mar 14 17:49 wheel
drwxr-sr-x. 2 root staff 4096 Mar 14 17:49 setuptools-38.5.2.dist-info
drwxr-sr-x. 5 root staff 4096 Mar 14 17:49 setuptools
drwxr-sr-x. 5 root staff 4096 Mar 14 17:49 pkg_resources
drwxr-sr-x. 2 root staff 4096 Mar 14 17:49 pip-9.0.1.dist-info
drwxr-sr-x. 11 root staff 4096 Mar 14 17:49 pip
-rw-r--r--. 1 root staff 126 Mar 14 17:49 easy_install.py
drwxr-sr-x. 1



+ ls -altr /usr/local/bin
total 188
drwxrwsr-x. 1 root staff 4096 Mar 12 00:00 ..
-rwxr-xr-x. 2 root staff 14312 Mar 14 17:49 python3.6m
-rwxr-xr-x. 2 root staff 14312 Mar 14 17:49 python3.6
-rwxr-xr-x. 1 root staff 441 Mar 14 17:49 pyvenv-3.6
lrwxrwxrwx. 1 root staff 10 Mar 14 17:49 pyvenv -> pyvenv-3.6
-rwxr-xr-x. 1 root staff 3109 Mar 14 17:49 python3.6m-config
lrwxrwxrwx. 1 root staff 17 Mar 14 17:49 python3.6-config -> python3.6m-config
lrwxrwxrwx. 1 root staff 16 Mar 14 17:49 python3-config -> python3.6-config
lrwxrwxrwx. 1 root staff 9 Mar 14 17:49 python3 -> python3.6
-rwxr-xr-x. 1 root staff 84 Mar 14 17:49 pydoc3.6
lrwxrwxrwx. 1 root staff 8 Mar 14 17:49 pydoc3 -> pydoc3.6
-rwxr-xr-x. 1 root staff 99 Mar 14 17:49 idle3.6
lrwxrwxrwx. 1 root staff 7 Mar 14 17:49 idle3 -> idle3.6
-rwxr-xr-x. 1 root staff 101 Mar 14 17:49 2to3-3.6
lrwxrwxrwx. 1 root staff 8 Mar 14 17:49 2to3 -> 2to3-3.6
lrwxrwxrwx. 1 root staff 14 Mar 14 17:49 python-config -> python3-config
lrwxrwxrwx. 1 root staff 7 Mar 14 17:49 python -> python3
lrwxrwxrwx. 1 root staff 6 Mar 14 17:49 pydoc -> pydoc3
lrwxrwxrwx. 1 root staff 5 Mar 14 17:49 idle -> idle3
-rwxr-xr-x. 1 root staff 218 Mar 14 17:49 wheel
-rwxr-xr-x. 1 root staff 211 Mar 14 17:49 pip3.6
-rwxr-xr-x. 1 root staff 211 Mar 14 17:49 pip3
-rwxr-xr-x. 1 root staff 211 Mar 14 17:49 pip
-rwxr-xr-x. 1 root staff 239 Mar 14 17:49 easy_install-3.6
-rwxr-xr-x. 1 r


+ which pip
/usr/local/bin/pip

+ which python
/usr/local/bin/python

 

But I cannot find the flask package (or any others for that matter!)

 

Please help!

 

Thanks,

Justin

1 answer

Since you are using python 3, shouldn't you use

pip3 install -r requirements.txt -vvv 

 

I assumed I would have to use pip as that is the cached dependancy in pipelines? It doesnt mention pip3....

I  updated the pip steps to use pip3 and it made no difference.

 

What I find most puzzling, is that at the end of the first step (pip3 install...)

this is the output:

changing mode of /usr/local/bin/flask to 755

 

So it definitely is there but, as you can see, Ive added that as a custom cache at the bottom, but the build still fails with the same error.

+ python -m flask run/usr/local/bin/python: No module named flask

I read each step creates a new docker image. Could it be related?

 

I can't get it to work either. I've got a pretty simple:

pipelines:
default:
- step:
name: Build
caches:
- pip
- setuptools

.......

definitions:
caches:
- setuptools: setuptools ~/.cache/setuptools

 

But at the end of each build, I always get:

Cache "setuptools": Skipping upload for empty cache

ank Atlassian Team Jul 26, 2020

Did we get any solution.

Suggest an answer

Log in or Sign up to answer
TAGS
Community showcase
Published in Bitbucket

New improvements to user management in Bitbucket Cloud 👥

Hey Community! We’re willing to wager that quite a few of you not only use Bitbucket, but administer it too. Our team is excited to share that we’ll be releasing improvements throughout this month of...

3,888 views 10 16
Read article

Community Events

Connect with like-minded Atlassian users at free events near you!

Find an event

Connect with like-minded Atlassian users at free events near you!

Unfortunately there are no Community Events near you at the moment.

Host an event

You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events

Events near you