Caching packages from docker builds

[sumanth]

I have pipeline as

definitions:
  services:
    docker:
      memory: 3072

pipelines:
  branches:
    build-*:
      - step:
          services:
            - docker
          caches:
            - docker
          image: atlassian/default-image:4
          oidc: true
          script:
            - git submodule update --init
            - git log -3 > .bitbucket/git-log.txt
            - docker build --build-arg COMMIT_ID=${BITBUCKET_COMMIT} -t my-image .
            - pipe: atlassian/aws-ecr-push-image:1.4.1
              variables:
                AWS_DEFAULT_REGION: ap-...
                AWS_OIDC_ROLE_ARN: arn:aws:iam::....
                IMAGE_NAME: my-image
                TAGS: $BITBUCKET_COMMIT
            - docker save -o tmp-image.name.tar my-image
          artifacts:
            - tmp-image.name.tar

And my Dockerfile has package installations as

RUN --mount=type=cache,id=pnpm,target=/tmp/pnpm pnpm install --frozen-lockfile
and
RUN --mount=type=cache,target=/tmp/uv,id=uv \
    --mount=type=bind,source=backend/uv.lock,target=${SERVER_DIR}/uv.lock \
    --mount=type=bind,source=backend/pyproject.toml,target=${SERVER_DIR}/pyproject.toml \
    uv sync --frozen --no-dev --no-install-project

I have tried many iterations of all suggestions I have found and am not able to make this work. My uv and pnpm tries to download during each build. Am I missing something here?

[Theodora Boudale · Accepted solution]

Hi @sumanth,

I'm afraid that there is a bug with the Docker cache at the moment which is not always working as expected. The Docker cache may sometimes work and other times not work at all. We have a bug report about this in our issue tracker:

• https://jira.atlassian.com/browse/BCLOUD-23193

You can add yourself as a watcher (by selecting the link Start watching this issue in the bug report) if you'd like to get notified via email on any updates. You can also select the link This affects my team to indicate you are affected by this issue.

If you see that the Docker cache is never used in your builds at all, I would suggest removing the Docker cache definition until the bug is fixed so that your build won't consume time downloading and uploading a cache that is not used.

Please also keep in mind that the following argument that you added to the RUN command of your Dockerfile won't help:

--mount=type=cache,id=pnpm,target=/tmp/pnpm

because every pipelines step is executed in a Docker container which gets destroyed once the step is finished.

Kind regards,
Theodora

[sumanth]

Thanks for the update

[Theodora Boudale]

You are very welcome.

Please feel free to reach out if you ever need anything else.

[sumanth]

Hi,
Just checked our CICD without the cache:docker and it seems to have increased our docker build size significantly. from (2.1GB to 2.5GB)
Before, the cache:docker was downloading artifact of around 600mb.

Is your note on 'If you see that the Docker cache is never used in your builds at all,...' correct or am I missing something?

[Theodora Boudale]

Hi @sumanth,

I can't say for sure what is happening in your builds without looking at your build logs, which I cannot do without a support ticket.

Since you have access to a workspace on a paid billing plan, you can create a ticket with the support team, share a build URL with the cache definition and one without the cache definition, point out where you see the difference and ask for help. A support ticket you create will be visible only to you and Atlassian staff, so anything you post there won't be publicly visible.

You can create a ticket via https://support.atlassian.com/contact/#/, in "What can we help you with?" select "Technical issues and bugs" and then Bitbucket Cloud as product. When you are asked to provide the workspace URL, please make sure you enter the URL of the workspace that is on a paid billing plan to proceed with ticket creation.

Kind regards,
Theodora