Docker cache is ignored

Question

My pipelines config:

image: php-fpm:7.3

pipelines:
branches:
master:
      - step:
name: Install dependecies
image: composer
script:
            - echo "Running composer in $(pwd)"
            - composer install --ignore-platform-reqs
caches:
            - composer
artifacts:
            - vendor/**
      - step:
name: Build and push to registry
image: docker
script:
            - docker info
            - docker login -u $DOCKER_HUB_USER -p $DOCKER_HUB_PASSWORD $DOCKER_HUB_HOST
            - docker build -t $DOCKER_HUB_HOST/km-backend:latest .
            - docker push $DOCKER_HUB_HOST/km-backend:latest
services:
            - docker
caches:
            - docker

On "Build and push to registry" step I can see the following messages:

Artifact "vendor/**": Downloading
Artifact "vendor/**": Downloaded 12.6 MiB in 3 seconds
Artifact "vendor/**": Extracting
Artifact "vendor/**": Extracted in 0 seconds
Cache "docker": Downloading
Cache "docker": Downloaded 252.7 MiB in 7 seconds
Cache "docker": Extracting
Cache "docker": Extracted in 4 seconds

But then it runs "docker build" command and ignores "docker" cache, installing everything again. My Dockerfile is pretty big and it contains many additional steps for building various PHP components, so it takes up to 7-8 minutes to build it after every commit. This is a very inefficient usage of resources, but I cannot figure out what am I doing wrong, so any help would be greatly appreciated.

Thank you.

vstokarev · Answer

Hi Philip. Thank you for the reply.

My Dockerfile:

FROM php:7.3-fpm

ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update && \
    apt-get -y install \
        gnupg2 && \
    apt-key update && \
    apt-get update && \
    apt-get -y install \
            g++ \
            git \
            curl \
            imagemagick \
            libcurl3-dev \
            libicu-dev \
            libfreetype6-dev \
            libjpeg-dev \
            libjpeg62-turbo-dev \
            libmagickwand-dev \
            libpq-dev \
            libpng-dev \
            libxml2-dev \
            libzip-dev \
            zlib1g-dev \
            mysql-client \
            openssh-client \
            nano \
            unzip \
            libcurl4-openssl-dev \
            libssl-dev \
--no-install-recommends && \
        apt-get clean && \
        rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

RUN docker-php-ext-configure gd \
--with-freetype-dir=/usr/include/ \
--with-png-dir=/usr/include/ \
--with-jpeg-dir=/usr/include/ && \
    docker-php-ext-configure bcmath && \
    docker-php-ext-install \
        soap \
        zip \
        curl \
        bcmath \
        exif \
        gd \
        iconv \
        intl \
        mbstring \
        opcache \
        pdo_mysql \
        pdo_pgsql \
        sockets

# Install PECL extensions
# see http://stackoverflow.com/a/8154466/291573) for usage of `printf`
RUN printf "
" | pecl config-set php_ini /usr/local/etc/php/php.ini && \
        pecl install \
        imagick \
        xdebug && \
    docker-php-ext-enable \
        imagick \
        xdebug

COPY . /application

WORKDIR /application

So seems like COPY is the reason why cache is invalidated according to that article you mentioned.

For the ADD and COPY instructions, the contents of the file(s) in the image are examined and a checksum is calculated for each file. The last-modified and last-accessed times of the file(s) are not considered in these checksums. During the cache lookup, the checksum is compared against the checksum in the existing images. If anything has changed in the file(s), such as the contents and metadata, then the cache is invalidated.

Well, it actually makes sense. I guess I'll just try to split my current 'build' step into two steps:

* Build PHP image.
* Build application image (this is where I put 'COPY' command).

Thanks for the hint!

Philip Hodder · Answer

Hi Vyacheslav,

Could you include a copy of your Dockerfile here?

When running "build" on Docker, once one Docker layer invalidates the Docker cache, all subsequent layers ignore the Docker cache. It's possible that you have a command early in your Dockerfile that invalidates the cache. See: https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#leverage-build-cache

Thanks,

Phil