How to specify cache for FTP step in bitbucket-pipelines.yml file?

Question

I understand that images can be cached to help reduce the run time of a pipeline, but I am not sure of the exact values that need to go in 'caches' section both in the 'step' and 'definitions'. For example, in this simple example to FTP files to a server, should I cache 'atlassian/default-image:2' and/or 'atlassian/ftp-deploy:0.3.6'? Do I need to cache 'docker'? Would like to know the exact text that goes in the YML file below. image: atlassian/default-image:2 pipelines: branches: master: - step: name: Deploy to Production deployment: production caches: - script: - pipe: atlassian/ftp-deploy:0.3.6 variables: USER: $FTP_USERNAME PASSWORD: $FTP_PASSWORD SERVER: $FTP_HOST REMOTE_PATH: / DELETE_FLAG: 'true' # Delete existing files EXTRA_ARGS: "--exclude=bitbucket-pipelines.yml --exclude=.gitignore" definitions: caches:

Theodora Boudale · Answer

Hi @mycall1234,

You are very welcome! What is still unclear to you? Is it how caching works?

In this specific example of the yaml file, when you use the pipe atlassian/ftp-deploy, the Docker image of the pipe needs to be downloaded from Dockerhub. If you set up cache, this Docker image will be saved in Atlassian infrastructure after the first successful build you run. The next time you run a build that uses this pipe, the Docker image of the pipe will be downloaded from the cache instead of connecting to Dockerhub and downloading it from there.

Please note that caches are saved for 1 week, after that they get cleared automatically and repopulated during the next build. You can read more info on the following documentation page and if you have any questions, please feel free to let me know:

* https://support.atlassian.com/bitbucket-cloud/docs/cache-dependencies/

You can check if the cache is available by going to the pipelines page of the repo and selecting the Caches option from the top right corner.

The improvement in run time may seem slow if the deploy itself takes more time. For example, let's say that without using cache, downloading the pipe's Docker image takes 6 seconds, and uploading the files to your server takes 3 minutes. If you use cache, what will be reduced is the 6 seconds that the pipe's Docker image takes to get downloaded from Dockerhub.

Kind regards,
Theodora

mycall1234 · Answer

Thank you, Theodora!  And sorry for the very late reply to your answer.

I made the changes you suggested. I don't see much improvement in run time, but perhaps that is because of the caching of all public Docker images.  I have to admit that it is not very clear to me how it works just yet.

Many thanks!

Theodora Boudale · Answer

Hi @mycall1234,

You can specify and get caching for the image atlassian/ftp-deploy. The yaml file should be as follows:

image: atlassian/default-image:2

pipelines:
  branches:
    master:
      - step:
          name: Deploy to Production
          deployment: production
          caches:
            - docker
          services:
            - docker
          script:
            - pipe: atlassian/ftp-deploy:0.3.6
              variables:
                USER: $FTP_USERNAME
                PASSWORD: $FTP_PASSWORD
                SERVER: $FTP_HOST
                REMOTE_PATH: /
                DELETE_FLAG: 'true' # Delete existing files
                EXTRA_ARGS: "--exclude=bitbucket-pipelines.yml --exclude=.gitignore"

You don't need to use the definitions section in the yaml file. The cache will be saved after the first successful build that executes this pipe, and it will be used in subsequent builds.

It is not possible to specify a cache for the Docker image of the build container (atlassian/default-image:2). However, we are caching all public Docker images from Dockerhub, so if you're using atlassian/default-image:2 or another public image from Dockerhub for the build container, a cache will be used. We also have a feature request for caching private Docker images for build/service containers here: https://jira.atlassian.com/browse/BCLOUD-19918

Kind regards,
Theodora