Can I keep Docker images between Pipeline steps?

[Engenharia Arquivei]

Hi, I am using the Pipeline to automate our builds and I've been struggling with the usage of steps and images. I am aware that I can keep generated files between steps using the artifacts option, but I can't seem to figure out how to keep docker images between steps.

My setup is currently like this:

pipelines:
  custom:
    dev:
      - step: 
          script:
            - # docker build
            - # push to GCR
            - # push to AWS ECR

 What I want is something like:

pipelines:
  custom:
    dev:
      - step: 
          script:
            - # docker build
          artifacts:
            - dist/**
      - step:
          script:
            - # push to GCR
      - step:
          script:
            - # push to AWS ECR 

Assume my credential configs are correct.

The problem is, tags created during the first step are not available on the next steps. So if I run docker build -t ${aws_ur}:${BITBUCKET_COMMIT} on the first step and then run docker push ${aws_url} on the last step the image will not exist (same for second step).

Am I doing something wrong (maybe the artifacts folder is wrong) or is there a way to do this?

Thanks in advance.

[nharquell]

For our project we were able to use docker save/load to share an image between steps.

- step:
  name: Build docker image
  script:
    - docker build -t "repo/imagename" .
    - docker save -output tmp-image.docker repo/imagename
  artifacts:
    - tmp-image.docker
- step:
  name: Deploy to Test
  deployment: test
  script:
   - docker load --input ./tmp-image.docker
   - docker images
   # repo/imagename should be available now

[Thomas Bunke]

Syntax correction:

- docker save --output

[kuzdimate]

doesnt work anymore :(

[Veronika Gnilitska]

worked for us, thanks

[Raphael Allner]

Hey guys. 

This solutions seems to work but I face some issues once I try to use the loaded image in the following steps within another docker build via

COPY --from=<imageTag>...

I always get following error:

Error processing tar file(exit status 1): Container ID 166537 cannot be mapped to a host ID

I already tried to use --chown=root:root option on COPY and a chown root:root on the first docker build of the <imageTag> taged image. Both without success. Does anybody have an idea? Is it just not possible to use the images between different docker build steps? 

[Andrew Crichton]

This is a couple years late, but maybe someone else encounters the same issue as Raphael. The issue doesn't have anything to do with persisting images between steps - this happens if you use `COPY --from...` without `export DOCKER_BUILDKIT=1` prior to the docker command (mentioned in the buildkit setup steps).

[André Ferro Cadete]

I figured it has to be the image size (limited to 1GB or not cacheable). I'd expect Bitbucket to cache all layers it can until 1GB is reached. For instance if I'm extending a PHP Docker image to install a few extensions and libs, I'd expect the base PHP Image to be cached if under 1GB. This is not the case it seems, it considers only the image we're building and mine easily exceeds 1GB since it also contains the codebase, tests and vendors after built.

Would be nice to see Bitbucket caching as many layers as possible before reaching 1GB for ANY image, not only the ones built (for most projects it's a lot more common the need to cache the BASE images we're extending).

[Xavier Serrat Bordas]

Is there any update on that? It would be nice to share docker images between steps to make pipelines faster when building multi-arch images. Due to the fact that bitbucket pipelines do not allow the use of the "--platform" flag for the "docker buildx build", the only way to build a docker images for both architectures is:

- parallel:

       - step: #Build image for arm64

       - step: #Build image for amd64

- step: # Final step to create the manifest and push it with both architectures

Thanks!!

[Rob Styles -DOWTK-]

Bump. Need an answer on this, or pipelines are useless/dangerous for dockerception builds.

[André Ferro Cadete]

Bump.. I'd love an answer in this one too :)