Is there a way to keep a pipeline cache of dependencies for more than one week?

Question

Hello, I'm not a bitbucket expert but I am here to learn.

is there a way to extend the bitbucket pipeline cache of external dependencies past one week?  We have external dependencies that won't change in the immediate future and some that cause the build to fail because they are intermittently off-line.

Or, is there a way to save the cache, possibly as an artifact(?), and download it to a new build, so that the cache doesn't need to be rebuilt?

Or, is there a way to use a docker container that has cached files already installed, and the pipeline can use these files, in place of the files that it would otherwise have to download?

Thank you in advance!

Theodora Boudale · Answer

Hi Gerald,

I am answering below the rest of your questions:

is there a way to save the cache, possibly as an artifact(?), and download it to a new build, so that the cache doesn't need to be rebuilt?

It is possible to define the directory of a dependency as an artifact, as long as it's downloaded in a directory inside BITBUCKET_CLONE_DIR (/opt/atlassian/pipelines/agent/build). However, artifacts can be used only by subsequent steps of the same build, they cannot be used in different builds.

is there a way to use a docker container that has cached files already installed, and the pipeline can use these files, in place of the files that it would otherwise have to download?

You could create a custom Docker image that has all the dependencies you need and use that as a build container for your builds, you can check the resources we have about that here:

* https://support.atlassian.com/bitbucket-cloud/docs/use-docker-images-as-build-environments/#Creating-a-custom-build-environment

Please note though that if these dependencies change in the future and you want to get the newer version, you would need to build that custom Docker image again so that it downloads the newest version of these dependencies and then push it again to the Docker registry where it is hosted (Dockerhub or another registry).

Please also note that most caches are saved in a directory relative to the clone path (BITBUCKET_CLONE_DIR) as you can see here:

* https://support.atlassian.com/bitbucket-cloud/docs/cache-dependencies/

If you create a custom Docker image that saves these dependencies in a different path, you may need to configure your project to find these dependencies in that different path.

Kind regards,
Theodora

Theodora Boudale · Answer

Hi @Gerald McCobb and welcome to the community!

At the moment, I'm afraid that it is not possible to keep a cache for more than a week. Any cache which is older than 1 week will be cleared automatically and repopulated during the next build.

When a cache is created, it will be used in subsequent builds (before the 1-week expiry) and the dependencies won't get downloaded again.

We do have a feature request to increase the expiry of the cache :

* https://jira.atlassian.com/browse/BCLOUD-15191

I would suggest adding your vote there (by selecting the Vote for this issue link) as the number of votes helps the development team and product managers better understand the demand for new features. You are more than welcome to leave any feedback, and you can also add yourself as a watcher (by selecting the Start watching this issue link) if you'd like to get notified via email on updates.

Implementation of new features is done as per our policy here and any updates will be posted in the feature request.

Please allow me to check with my team regarding your third question and I'll get back to you.

Kind regards,
Theodora