Come for the products,
stay for the community

The Atlassian Community can help you and your team get more value out of Atlassian products and practices.

Atlassian Community about banner
4,369,454
Community Members
 
Community Events
168
Community Groups

How do the caches work for Pipeline Runners?

Hi,

I'm running a group of tests with the Runner feature of Bitbucket Pipelines. Inside the runners I have steps to install pip requirements for testing and have specified pip as a cache for the runners steps. However the runners don't seem to be pulling these requirements from cache on subsequent runs.

Can anyone point out where I'm going wrong in the high level configuration below:

pull-requests:
"**":
- parallel:
- step:
name: solution_testing_0
runs-on:
- self.hosted
- linux
- solution.testing
caches:
- pip
script:
- bash install_system_requirements.sh # Installs all pip requirements
- bash run_test.sh --group 0 # Runs subset of tests
- step:
name: solution_testing_1
runs-on:
- self.hosted
- linux
- solution.testing
caches:
- pip
script:
- bash install_system_requirements.sh # Installs all pip requirements
- bash run_test.sh --group 1 # Runs subset of tests


How are caches supposed to be configured for pipeline runners?

1 answer

1 accepted

0 votes
Answer accepted

@ntmat Caches work the same way for both cloud and self-hosted runners. Be aware that the performance improvement from caches is different between cloud and self-hosted runners. Self-hosted runners are located in your data center, so it might take more time to pull the cache from Bitbucket. 

@Justin ThomasThanks for your reply. I can confirm that the cloud and self-hosted runners use the cache as expected, I needed to delete the existing cache so the runner would re-upload and use the new pip cache.

I see your point regarding performance, what is the alternative? Would you recommend using a docker:dind service to startup a local docker image with all the necessary dependencies preinstalled to avoid cache download from Bitbucket for self.hosted runners?

Unfortunately, there is no easy way around it. Using a docker:dind service is a workaround and can help, but I think in long term it might complicate the setup. It depends on the performance impact.

Cheers,

Justin

Suggest an answer

Log in or Sign up to answer
TAGS

Atlassian Community Events