We are using runners as a way of running long tests without using up a lot of build minutes. As part of these runners, we are using pipes to run SSH commands and do SCP deploys.
We don't have a fast internet connection to the machine that is hosting the runners, so each time we run a new runner it takes a few minutes to download the pipe.
Is there a way of caching pipes between runs so that we don't have to download the whole pipe each time we have a new pipeline run?
It is possible to use cache with runners, the cache will be uploaded to Atlassian infrastructure though and not locally. However, since the cache is then downloaded compressed, it may still be faster than not using cache (since without using cache, each layer of the pipe's docker image will be downloaded).
If you want to try it, you can add the following in your yml file, to each step that is using these pipes:
The cache for each pipe's docker image will be uploaded after the first successful build that is using each pipe. The next time you use a pipe, the docker image of the pipe will be downloaded from the cache.
If this is also taking a long time, another option would be to use SSH/SCP commands in your script to connect to your server and run commands/deploy, instead of using pipes.
Beginning on April 4th, we will be implementing push limits. This means that your push cannot be completed if it is over 3.5 GB. If you do attempt to complete a push that is over 3.5 GB, it will fail...
Connect with like-minded Atlassian users at free events near you!Find an event
Connect with like-minded Atlassian users at free events near you!
Unfortunately there are no Community Events near you at the moment.Host an event
You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events