[Pipelines] How to build a Docker image with a FROM pointing to a private image



Here's our use case: as our binaries are build from a big SDK, building it from scratch takes 20 min. 

But the most frequent changes can be built incrementally (and therefore the build is a few seconds, up to 1 or 2 minutes)

Here's what we're trying to achieve:

- We have a base build image (quite large) with the base SDK and the build tools, which we update once in a while. We run the build script locally and push it to ECR (since some paths needs to be manually changed anyway when the SDK gets updated), the private image ends like 123456789012.dkr.ecr.us-east-1.amazonaws.com/imagename:sdk_v1


- Then we have some update layer on top of the sdk, which is updated from time to time. These files are added to the SDK then we build it. This is the step which takes 20 min


- Finally, we have our main sources, which uses the output of the previous step. This one is frequently updated (and build from the previous step is fast)


Previously, as a first step into Bitbucket pipelines, we ran step 2 and 3 together, using the image from step1 as our build env. This was running fine, but builds are taking a long time although only the small part which is fast to build is updated. So we plan to split it like described above.

The idea was in step 2 to do a docker build, and the Dockerfile has a `FROM 

123456789012.dkr.ecr.us-east-1.amazonaws.com/imagename:sdk_v1` directive so that we can produce the final image from there. 

But I realized that the auth is correct to use private image as build environment, but although this auth is set, the base image cannot be pulled from ECR, and I get errors like



Sending build context to Docker daemon 244.2kB
Step 1/4 : FROM 123456789012.dkr.ecr.us-east-1.amazonaws.com/imagename:sdk_v1
Pulling repository 123456789012.dkr.ecr.us-east-1.amazonaws.com/imagename
unauthorized: authentication required



Since our build image has everything we need, I tried to use it as the build environment at the same time, to make sure the authentication is set before the pull... and although the build environment is correct, it seems that the auth used for the build environment isn't available for pulling the base image when doing the docker build.

Here's the top of our bitbucket-pipelines.yml (with values changed for anonymizing a little bit)


  name: 123456789012.dkr.ecr.us-east-1.amazonaws.com/imagename:sdk_v1
      access-key: $AWS_ACCESS_KEY_ID
      secret-key: $AWS_SECRET_ACCESS_KEY



I'm trying to find a workaround, but if someone can spot some step I missed to correctly use a private ECR image in a `FROM` directive when doing a Docker build, it would be really awesome to solve our issue and get fast builds!



0 answers

Suggest an answer

Log in or Join to answer
Community showcase
Piotr Plewa
Published Dec 27, 2017 in Bitbucket

Recipe: Deploying AWS Lambda functions with Bitbucket Pipelines

Bitbucket Pipelines helps me manage and automate a number of serverless deployments to AWS Lambda and this is how I do it. I'm building Node.js Lambda functions using node-lambda&nbsp...

673 views 0 4
Read article

Atlassian User Groups

Connect with like-minded Atlassian users at free events near you!

Find a group

Connect with like-minded Atlassian users at free events near you!

Find my local user group

Unfortunately there are no AUG chapters near you at the moment.

Start an AUG

You're one step closer to meeting fellow Atlassian users at your local meet up. Learn more about AUGs

Groups near you
Atlassian Team Tour

Join us on the Team Tour

We're bringing product updates and pro tips on teamwork to ten cities around the world.

Save your spot