nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Docker Cache not working

Michael Maier_s2sgf6

I have this bitbucket-pipelines.yml

image: atlassian/default-image

pipelines:
  custom:
    test-build-docker-image:
      - step:
          name: Build Docker image
          script:
            - |
              docker build -f - . <<EOF
              FROM busybox:1.36.1
              CMD echo just a test
              EOF
          caches:
            - docker
          services:
            - docker

definitions:
  services:
    docker:
      memory: 7168

options:
  docker: true #enabling docker daemon
  size: 2x

When I run the pipeline the second time, I expect the layers to be cached. However, the pipeline always builds from scratch (incl. pulling the busybox image).

Output of the first run:

Output of the second run:

PS: The editor of this form sucks. I can't paste code without loosing indentation.

Accepted answers

All comments

Theodora Boudale

Hi Michael!

I ran a few builds using the same bitbucket-pipelines.yml that you shared here and the Docker cache is working as expected.

Looking at the output of the first run, in the Build teardown section, we can see that no cache was uploaded from this build because a Docker cache was already present in the repo.

Did you previously run a different docker build that generated a cache? Or do you have another pipeline on a different branch of the repo that generated a Docker cache?

Can you try clearing the cache (from the Pipelines page of the repo, option Caches at the top right corner) and running two more builds (one to generate the cache and the second to confirm if it's used)?

Kind regards,
Theodora

Michael Maier_s2sgf6

Yes, I ran a pipeline for a different Dockerfile before. Do I need to use different caches for each Dockerfile? What if the Dockerfile changes? Are new layers not added to the cache?

For example, if I build

FROM busybox:1.36.1
RUN echo "first"
RUN echo "second"
RUN echo "third"

twice, I expect all RUN layers to be cached in the second run. If I change "second" to "foo", I expect "first" to be cached, but not "third", since previous layer changed:

FROM busybox:1.36.1
RUN echo "first"
RUN echo "foo"
RUN echo "third"

After I build the changed Dockerfile (above), I expect the cache to be updated, i.e. if I build the changed Dockerfile another time, all layers should be cached.

Is this the case?

Theodora Boudale

Hi Michael,

If I change "second" to "foo", I expect "first" to be cached, but not "third", since previous layer changed

With the Docker cache definition you are using, this should work as you described. I tried it as well and "first" is using cache, while "foo" and "third" do not.

However, the cache does not get updated by default when there are changes. In order for this to happen, we need to use caching with file-based cache keys:

https://support.atlassian.com/bitbucket-cloud/docs/cache-dependencies/#Caching-with-file-based-cache-keys

For Docker, this means that we need to use a custom cache definition since the default one does not offer the path variable that is required.

If the Dockerfile is committed to the repo, you can use a yml file as follows:

image: atlassian/default-image:4

pipelines:
  custom:
    test-build-docker-image:
      - step:
          name: Build Docker image
          script:
            - docker load -i docker-cache/* || echo "No cache"
            - docker build .
            - mkdir -p docker-cache && docker save $(docker images -aq) -o docker-cache/cache.tar
          caches:
            - my-docker-cache
          services:
            - docker

definitions:
  services:
    docker:
      memory: 7168
  caches:
    my-docker-cache:
      key:
        files:
          - Dockerfile
      path: docker-cache

options:
  size: 2x

With a yml file like this, the cache will be updated every time there is a change in the Dockerfile that is committed to the repo.

Please keep in mind that in this case, the following will not work:

If I change "second" to "foo", I expect "first" to be cached, but not "third", since previous layer changed

If there is going to be a change in the Dockerfile, the build is not going to use the previous cache at all.

Please feel free to let me know if you have any questions.

Kind regards,
Theodora

maiermic

Thanks, that should be more usable. However, it is a lot more hassle to configure. What about build arguments?

Welcome!

It looks like you're new here. Sign in or register to get started.

Quick Links

Find more posts tagged with

cloud

bitbucket

bitbucket-cloud

bitbucket-pipelines