It's not the same without you

Join the community to find out what other Atlassian users are discussing, debating and creating.

Atlassian Community Hero Image Collage

Docker cache is ignored Edited

My pipelines config:

image: php-fpm:7.3

pipelines:
branches:
master:
- step:
name: Install dependecies
image: composer
script:
- echo "Running composer in $(pwd)"
- composer install --ignore-platform-reqs
caches:
- composer
artifacts:
- vendor/**
- step:
name: Build and push to registry
image: docker
script:
- docker info
- docker login -u $DOCKER_HUB_USER -p $DOCKER_HUB_PASSWORD $DOCKER_HUB_HOST
- docker build -t $DOCKER_HUB_HOST/km-backend:latest .
- docker push $DOCKER_HUB_HOST/km-backend:latest
services:
- docker
caches:
- docker

On "Build and push to registry" step I can see the following messages:

Artifact "vendor/**": Downloading
Artifact "vendor/**": Downloaded 12.6 MiB in 3 seconds
Artifact "vendor/**": Extracting
Artifact "vendor/**": Extracted in 0 seconds
Cache "docker": Downloading
Cache "docker": Downloaded 252.7 MiB in 7 seconds
Cache "docker": Extracting
Cache "docker": Extracted in 4 seconds

But then it runs "docker build" command and ignores "docker" cache, installing everything again. My Dockerfile is pretty big and it contains many additional steps for building various PHP components, so it takes up to 7-8 minutes to build it after every commit. This is a very inefficient usage of resources, but I cannot figure out what am I doing wrong, so any help would be greatly appreciated.

Thank you.

1 answer

0 votes

Hi Vyacheslav,

Could you include a copy of your Dockerfile here?

When running "build" on Docker, once one Docker layer invalidates the Docker cache, all subsequent layers ignore the Docker cache. It's possible that you have a command early in your Dockerfile that invalidates the cache. See: https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#leverage-build-cache

Thanks,

Phil

Hi Philip. Thank you for the reply.

My Dockerfile:

FROM php:7.3-fpm

ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update && \
apt-get -y install \
gnupg2 && \
apt-key update && \
apt-get update && \
apt-get -y install \
g++ \
git \
curl \
imagemagick \
libcurl3-dev \
libicu-dev \
libfreetype6-dev \
libjpeg-dev \
libjpeg62-turbo-dev \
libmagickwand-dev \
libpq-dev \
libpng-dev \
libxml2-dev \
libzip-dev \
zlib1g-dev \
mysql-client \
openssh-client \
nano \
unzip \
libcurl4-openssl-dev \
libssl-dev \
--no-install-recommends && \
apt-get clean && \
rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

RUN docker-php-ext-configure gd \
--with-freetype-dir=/usr/include/ \
--with-png-dir=/usr/include/ \
--with-jpeg-dir=/usr/include/ && \
docker-php-ext-configure bcmath && \
docker-php-ext-install \
soap \
zip \
curl \
bcmath \
exif \
gd \
iconv \
intl \
mbstring \
opcache \
pdo_mysql \
pdo_pgsql \
sockets

# Install PECL extensions
# see http://stackoverflow.com/a/8154466/291573) for usage of `printf`
RUN printf "\n" | pecl config-set php_ini /usr/local/etc/php/php.ini && \
pecl install \
imagick \
xdebug && \
docker-php-ext-enable \
imagick \
xdebug

COPY . /application

WORKDIR /application

So seems like COPY is the reason why cache is invalidated according to that article you mentioned.

For the ADD and COPY instructions, the contents of the file(s) in the image are examined and a checksum is calculated for each file. The last-modified and last-accessed times of the file(s) are not considered in these checksums. During the cache lookup, the checksum is compared against the checksum in the existing images. If anything has changed in the file(s), such as the contents and metadata, then the cache is invalidated.

Well, it actually makes sense. I guess I'll just try to split my current 'build' step into two steps:

  1. Build PHP image.
  2. Build application image (this is where I put 'COPY' command).

Thanks for the hint!

Suggest an answer

Log in or Sign up to answer
TAGS
Community showcase
Published in Bitbucket Pipelines

What We Learned When We Researched Open Source Vulnerabilities in 7 Popular Coding Languages

...hey are a part of us, shaping how we interact with the world around us. The same holds true for programming languages when we think about how different kinds of vulnerabilities raise their heads in t...

1,044 views 0 3
Read article

Community Events

Connect with like-minded Atlassian users at free events near you!

Find an event

Connect with like-minded Atlassian users at free events near you!

Unfortunately there are no Community Events near you at the moment.

Host an event

You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events

Events near you