How to cache pipe on a local runner

Ashley Unitt_6m4997

We are using runners as a way of running long tests without using up a lot of build minutes. As part of these runners, we are using pipes to run SSH commands and do SCP deploys.

We don't have a fast internet connection to the machine that is hosting the runners, so each time we run a new runner it takes a few minutes to download the pipe.

Is there a way of caching pipes between runs so that we don't have to download the whole pipe each time we have a new pipeline run?

Theodora Boudale

Hi Ashley,

It is possible to use cache with runners, the cache will be uploaded to Atlassian infrastructure though and not locally. However, since the cache is then downloaded compressed, it may still be faster than not using cache (since without using cache, each layer of the pipe's docker image will be downloaded).

If you want to try it, you can add the following in your yml file, to each step that is using these pipes:

services:
  - docker
caches:
  - docker

The cache for each pipe's docker image will be uploaded after the first successful build that is using each pipe. The next time you use a pipe, the docker image of the pipe will be downloaded from the cache.

If this is also taking a long time, another option would be to use SSH/SCP commands in your script to connect to your server and run commands/deploy, instead of using pipes.

Kind regards,
Theodora