Docker cache is ignored

[vstokarev]

My pipelines config:

image: php-fpm:7.3

pipelines:
  branches:
    master:
      - step:
          name: Install dependecies
          image: composer
          script:
            - echo "Running composer in $(pwd)"
            - composer install --ignore-platform-reqs
          caches:
            - composer
          artifacts:
            - vendor/**
      - step:
          name: Build and push to registry
          image: docker
          script:
            - docker info
            - docker login -u $DOCKER_HUB_USER -p $DOCKER_HUB_PASSWORD $DOCKER_HUB_HOST
            - docker build -t $DOCKER_HUB_HOST/km-backend:latest .
            - docker push $DOCKER_HUB_HOST/km-backend:latest
          services:
            - docker
          caches:
            - docker

On "Build and push to registry" step I can see the following messages:

Artifact "vendor/**": Downloading
Artifact "vendor/**": Downloaded 12.6 MiB in 3 seconds
Artifact "vendor/**": Extracting
Artifact "vendor/**": Extracted in 0 seconds
Cache "docker": Downloading
Cache "docker": Downloaded 252.7 MiB in 7 seconds
Cache "docker": Extracting
Cache "docker": Extracted in 4 seconds

But then it runs "docker build" command and ignores "docker" cache, installing everything again. My Dockerfile is pretty big and it contains many additional steps for building various PHP components, so it takes up to 7-8 minutes to build it after every commit. This is a very inefficient usage of resources, but I cannot figure out what am I doing wrong, so any help would be greatly appreciated.

Thank you.

[Philip Hodder]

Hi Vyacheslav,

Could you include a copy of your Dockerfile here?

When running "build" on Docker, once one Docker layer invalidates the Docker cache, all subsequent layers ignore the Docker cache. It's possible that you have a command early in your Dockerfile that invalidates the cache. See: https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#leverage-build-cache

Thanks,

Phil

[vstokarev]

Hi Philip. Thank you for the reply.

My Dockerfile:

FROM php:7.3-fpm

ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update && \
    apt-get -y install \
        gnupg2 && \
    apt-key update && \
    apt-get update && \
    apt-get -y install \
            g++ \
            git \
            curl \
            imagemagick \
            libcurl3-dev \
            libicu-dev \
            libfreetype6-dev \
            libjpeg-dev \
            libjpeg62-turbo-dev \
            libmagickwand-dev \
            libpq-dev \
            libpng-dev \
            libxml2-dev \
            libzip-dev \
            zlib1g-dev \
            mysql-client \
            openssh-client \
            nano \
            unzip \
            libcurl4-openssl-dev \
            libssl-dev \
        --no-install-recommends && \
        apt-get clean && \
        rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

RUN docker-php-ext-configure gd \
        --with-freetype-dir=/usr/include/ \
        --with-png-dir=/usr/include/ \
        --with-jpeg-dir=/usr/include/ && \
    docker-php-ext-configure bcmath && \
    docker-php-ext-install \
        soap \
        zip \
        curl \
        bcmath \
        exif \
        gd \
        iconv \
        intl \
        mbstring \
        opcache \
        pdo_mysql \
        pdo_pgsql \
        sockets

# Install PECL extensions
# see http://stackoverflow.com/a/8154466/291573) for usage of `printf`
RUN printf "\n" | pecl config-set php_ini /usr/local/etc/php/php.ini && \
        pecl install \
        imagick \
        xdebug && \
    docker-php-ext-enable \
        imagick \
        xdebug

COPY . /application

WORKDIR /application

So seems like COPY is the reason why cache is invalidated according to that article you mentioned.

For the ADD and COPY instructions, the contents of the file(s) in the image are examined and a checksum is calculated for each file. The last-modified and last-accessed times of the file(s) are not considered in these checksums. During the cache lookup, the checksum is compared against the checksum in the existing images. If anything has changed in the file(s), such as the contents and metadata, then the cache is invalidated.

Well, it actually makes sense. I guess I'll just try to split my current 'build' step into two steps:

1. Build PHP image.
2. Build application image (this is where I put 'COPY' command).

Thanks for the hint!