Come for the products,
stay for the community

The Atlassian Community can help you and your team get more value out of Atlassian products and practices.

Atlassian Community about banner
4,298,556
Community Members
 
Community Events
165
Community Groups

How to specify cache for FTP step in bitbucket-pipelines.yml file?

I understand that images can be cached to help reduce the run time of a pipeline, but I am not sure of the exact values that need to go in 'caches' section both in the 'step' and 'definitions'. For example, in this simple example to FTP files to a server, should I cache 'atlassian/default-image:2' and/or 'atlassian/ftp-deploy:0.3.6'? Do I need to cache 'docker'?  Would like to know the exact text that goes in the YML file below.

image: atlassian/default-image:2

pipelines:
  branches:
    master:
      - step:
          name: Deploy to Production
          deployment: production
          caches:
             - <WHAT GOES HERE?>
          script:
            - pipe: atlassian/ftp-deploy:0.3.6
                 variables:
                   USER: $FTP_USERNAME
                   PASSWORD: $FTP_PASSWORD
                   SERVER: $FTP_HOST
                   REMOTE_PATH: /
                   DELETE_FLAG: 'true' # Delete existing files
                   EXTRA_ARGS: "--exclude=bitbucket-pipelines.yml --exclude=.gitignore"
definitions:
  caches:
    <WHAT GOES HERE?>

1 answer

0 votes

Hi @mycall1234,

You can specify and get caching for the image atlassian/ftp-deploy. The yaml file should be as follows:

image: atlassian/default-image:2

pipelines:
branches:
master:
- step:
name: Deploy to Production
deployment: production
caches:
- docker
services:
- docker
script:
- pipe: atlassian/ftp-deploy:0.3.6
variables:
USER: $FTP_USERNAME
PASSWORD: $FTP_PASSWORD
SERVER: $FTP_HOST
REMOTE_PATH: /
DELETE_FLAG: 'true' # Delete existing files
EXTRA_ARGS: "--exclude=bitbucket-pipelines.yml --exclude=.gitignore"

You don't need to use the definitions section in the yaml file. The cache will be saved after the first successful build that executes this pipe, and it will be used in subsequent builds.

It is not possible to specify a cache for the Docker image of the build container (atlassian/default-image:2). However, we are caching all public Docker images from Dockerhub, so if you're using atlassian/default-image:2 or another public image from Dockerhub for the build container, a cache will be used. We also have a feature request for caching private Docker images for build/service containers here: https://jira.atlassian.com/browse/BCLOUD-19918

Kind regards,
Theodora

Thank you, Theodora!  And sorry for the very late reply to your answer.

I made the changes you suggested. I don't see much improvement in run time, but perhaps that is because of the caching of all public Docker images.  I have to admit that it is not very clear to me how it works just yet.

Many thanks!

Hi @mycall1234,

You are very welcome! What is still unclear to you? Is it how caching works?

In this specific example of the yaml file, when you use the pipe atlassian/ftp-deploy, the Docker image of the pipe needs to be downloaded from Dockerhub. If you set up cache, this Docker image will be saved in Atlassian infrastructure after the first successful build you run. The next time you run a build that uses this pipe, the Docker image of the pipe will be downloaded from the cache instead of connecting to Dockerhub and downloading it from there.

Please note that caches are saved for 1 week, after that they get cleared automatically and repopulated during the next build. You can read more info on the following documentation page and if you have any questions, please feel free to let me know:

You can check if the cache is available by going to the pipelines page of the repo and selecting the Caches option from the top right corner.

The improvement in run time may seem slow if the deploy itself takes more time. For example, let's say that without using cache, downloading the pipe's Docker image takes 6 seconds, and uploading the files to your server takes 3 minutes. If you use cache, what will be reduced is the 6 seconds that the pipe's Docker image takes to get downloaded from Dockerhub.

Kind regards,
Theodora

Suggest an answer

Log in or Sign up to answer
TAGS
Community showcase
Published in Bitbucket

Git push size limits are coming to Bitbucket Cloud starting April 4th, 2022

Beginning on April 4th, we will be implementing push limits. This means that your push cannot be completed if it is over 3.5 GB. If you do attempt to complete a push that is over 3.5 GB, it will fail...

2,231 views 2 9
Read article

Community Events

Connect with like-minded Atlassian users at free events near you!

Find an event

Connect with like-minded Atlassian users at free events near you!

Unfortunately there are no Community Events near you at the moment.

Host an event

You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events

Events near you