Create
cancel
Showing results for 
Search instead for 
Did you mean: 
Sign up Log in
It's not the same without you

Join the community to find out what other Atlassian users are discussing, debating and creating.

Atlassian Community Hero Image Collage

Created a pipe to use in next steps, but installed pip pakcages not found

Edited

Hi everyone, so my pipeline looks like this:

image: atlassian/default-image:2
solved

1 answer

0 votes

@Elgiz Abbasov Hi. Happy that you solved your issue. 

To discover more pipes, visit our Bitbucket Pipes Marketplace.

Regards, Igor. 

Hey Igor, just a quick question as a follow up, I am using a Dockerfile that has some packages installing using apt-get. In my BB pipelines even though I have set up Docker as my cache, it only caches the first 3 steps which are the least time consuming (env var setup) and the apt-get installs are not cached at all. Do you know how can I fix this so that I don't have to build the image from scratch in my pipeline step each time? Reading online people said Dockerfile should not contain apt-get installs (that there is a better way?) I know I can use DockerHub and update the base image everytime but I don't want to do that. Currently my pipeline takes ~10 minutes and it doesn't cache the Docker pip downloads using pipenv nor does it cache the apt-get installs. Any help would be greatly areciated

@Elgiz Abbasov Hi. Please provide us your Dockerfile so we could help you.

Regards, Igor.

@Igor Stoyanov My Dockerfile:

FROM python:3.9

# Set environment variables

ENV PYTHONDONTWRITEBYTECODE 1

ENV PYTHONUNBUFFERED 1

ENV PYTHONPATH "${PYTHONPATH}:/opt/atlassian/pipelines/agent/build"

# Set work directory

WORKDIR /code

# Install dependencies

COPY Pipfile Pipfile.lock /code/

RUN pip install pipenv && pipenv install --dev --system

RUN apt-get update && apt-get autoclean && \

apt-get install -y '^libxcb.*-dev' libx11-xcb-dev libglu1-mesa-dev libxrender-dev libxi-dev libxcb-xinerama0 libxkbcommon-dev libxkbcommon-x11-dev libxkbcommon-x11-0 libdbus-1-3 xvfb

# Copy project

COPY . /code/

@Igor Stoyanov My partial bitbucket-pipelines.yml file:

It builds the image from scratch every time (only caching env vars in Dockerfile) and taking a lot of time to install apt-get and pip packages in Dockerfile. I thought adding caches: docker would cache the image but it doesn't, instead the image is built and saved in tmp-image.docker (from scratch everytime). I want it so that if Pipfile or Dockerfile hasn't changed, it just uses the tmp-image.docker from previous pipeline run, but again caches: docker didn't help me with that.

 

image: atlassian/default-image:2

options:

docker: true

pipelines:

pull-requests:

'**': #this runs as default for any branch not elsewhere defined

- step:

name: 'Build Docker and Setup'

caches:

- docker

- pip

script:

- docker build -t elgiz/app .

- docker save --output tmp-image.docker elgiz/app

artifacts:

- tmp-image.docker

- parallel:

- step:

name: 'PyTest'

script:

- docker load --input ./tmp-image.docker

- echo "PyTest running..."

- docker run --init -v $BITBUCKET_CLONE_DIR:/opt/atlassian/pipelines/agent/build/app elgiz/app xvfb-run pytest --cov=test/

- echo "PyTest completed."

@Elgiz Abbasov hi.

Maybe its good idea:

  1. to create your own docker image with all needed dependencies
  2. or to find in docker hub image that already has most of needed dependencies that you need

Regards, Igor

@Igor Stoyanov The goal was to avoid using DockerHub and use bitbucket pipelines features but I guess the caching doesn't work as I expected ;/ Thanks

Suggest an answer

Log in or Sign up to answer
TAGS
Community showcase
Published in Bitbucket

📣 Calling Bitbucket Data Center customers to participate in research

Hi everyone, Are you Bitbucket DC customer? If so, we'd love to talk to you! Our team wants to dive deep to understand your long-term plans regarding Bitbucket DC and Atlassian Cloud. Do you plan...

209 views 2 5
Read article

Community Events

Connect with like-minded Atlassian users at free events near you!

Find an event

Connect with like-minded Atlassian users at free events near you!

Unfortunately there are no Community Events near you at the moment.

Host an event

You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events

Events near you