Create
cancel
Showing results for 
Search instead for 
Did you mean: 
Sign up Log in

It's not the same without you

Join the community to find out what other Atlassian users are discussing, debating and creating.

Atlassian Community Hero Image Collage

How to configure the cache for the conda package manager in bitbucket pipelines

Hi all,

I use the following pipeline for checking the source code:

 

# This is a sample build configuration for Python.
# Check our guides at https://confluence.atlassian.com/x/x4UWN for more examples.
# Only use spaces to indent your .yml configuration.
# -----
# You can specify a custom docker image from Docker Hub as your build environment.
image: continuumio/miniconda3:latest

pipelines:
default:
- step:
script: # Modify the commands below to build your repository.
- conda env update -n root --file environment.yml
- pylint module1
- mypy -m module1 --ignore-missing-imports --strict
- coverage run --source=module1 -m unittest discover
- coverage report -m

I want to use the bitbucket caches to make the pipeline faster but i am not sure which directories i should cache.

I already asked the question at the conda user group   but did not get an answer.

How can i configure the cache ?

 

Thanks,

David

2 answers

1 accepted

1 vote
Answer accepted
image: python:3.6.2
pipelines:
default:
- step:
caches:
- condacache
script:
- /opt/python/bin/conda update -y conda
- /opt/python/bin/pip install --upgrade pip
- /opt/python/bin/pip install -r requirements.txt
- /opt/python/bin/pytest

definitions:
caches:
condacache: /opt/python/bin


#This is how i used to create cache in my pipeline: Please take it as reference.
#This is a custom cache technique. Hope it would help you.

This does not work.

When using the `continuumio/miniconda3` image in your pipeline, I have found that defining a `conda` cache as `/opt/conda/pkgs` worked. After the first run of the pipeline, in subsequent runs of the pipeline the command `conda env update` should not longer print "Downloading and Extracting Packages"

This does not work for me.

image: continuumio/miniconda3:latest

pipelines:
default:
- step:
caches:
- conda
- codatarballs

script:
- conda env update -n test_env --file environment.yml

definitions:
caches:
conda: ~/.conda/envs/test_env
codatarballs: /opt/conda/pkgs

The output I get is:

Assembling contents of new cache 'conda'
Assembling contents of new cache 'codatarballs'

but then on the next run:

Cache "conda": Downloading
Cache "conda": Not found
Cache "codatarballs": Downloading
Cache "codatarballs": Not found

I don't know why it doesn't download the cache in the next run, it does for me, and I have the same output at the end of the first run:

Assembling contents of new cache 'conda'

One thing you can check is in the main pipelines screen of the repository, there should be a button on the top right named "Caches". If I click that I see:

conda: opt/conda/pkgs 350.31 MB 29 May 2020

Hope this helps!

I've open an ticket 'cos it makes no sense. maybe they have some more logs

Suggest an answer

Log in or Sign up to answer
TAGS
Community showcase
Published in Bitbucket

New improvements to user management in Bitbucket Cloud 👥

Hey Community! We’re willing to wager that quite a few of you not only use Bitbucket, but administer it too. Our team is excited to share that we’ll be releasing improvements throughout this month of...

3,734 views 10 16
Read article

Community Events

Connect with like-minded Atlassian users at free events near you!

Find an event

Connect with like-minded Atlassian users at free events near you!

Unfortunately there are no Community Events near you at the moment.

Host an event

You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events

Events near you