Come for the products,
stay for the community

The Atlassian Community can help you and your team get more value out of Atlassian products and practices.

Atlassian Community about banner
Community Members
Community Events
Community Groups

How to cache pipe on a local runner

We are using runners as a way of running long tests without using up a lot of build minutes. As part of these runners, we are using pipes to run SSH commands and do SCP deploys.

We don't have a fast internet connection to the machine that is hosting the runners, so each time we run a new runner it takes a few minutes to download the pipe.

Is there a way of caching pipes between runs so that we don't have to download the whole pipe each time we have a new pipeline run?

1 answer

0 votes

Hi Ashley,

It is possible to use cache with runners, the cache will be uploaded to Atlassian infrastructure though and not locally. However, since the cache is then downloaded compressed, it may still be faster than not using cache (since without using cache, each layer of the pipe's docker image will be downloaded).

If you want to try it, you can add the following in your yml file, to each step that is using these pipes:

- docker
- docker

The cache for each pipe's docker image will be uploaded after the first successful build that is using each pipe. The next time you use a pipe, the docker image of the pipe will be downloaded from the cache.

If this is also taking a long time, another option would be to use SSH/SCP commands in your script to connect to your server and run commands/deploy, instead of using pipes.

Kind regards,

Suggest an answer

Log in or Sign up to answer
Community showcase
Published in Bitbucket

Git push size limits are coming to Bitbucket Cloud starting April 4th, 2022

Beginning on April 4th, we will be implementing push limits. This means that your push cannot be completed if it is over 3.5 GB. If you do attempt to complete a push that is over 3.5 GB, it will fail...

2,249 views 2 9
Read article

Community Events

Connect with like-minded Atlassian users at free events near you!

Find an event

Connect with like-minded Atlassian users at free events near you!

Unfortunately there are no Community Events near you at the moment.

Host an event

You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events

Events near you