Pipelines: Pulling docker images from a Amazon ECR repository

For a build pipeline I would like to start from a docker image stored in a Amazon EC2 Container Registry (ECR) repository. Pulling this image would normally require me to do a 'aws ecr get-login' first, but I fail to see how this would work with a in a bitbucket-pipelines.yml file. How do I get this working,? Since the pull is the first step of the build pipeline, I don't think I can execute a script?

 

Thx

4 answers

2 votes

It looks like Amazon ECR currently only allows you to generate Docker credentials that are valid for 12 hours: http://docs.aws.amazon.com/cli/latest/reference/ecr/get-authorization-token.html

Generating these for your build should work, however I suppose it is quite inconvenient to regenerate them all of the time. 

I've just outlined a potential workaround, only to stumble over the apparent lack of an actual Bitbucket Pipelines API - presumably this is just not officially documented yet (the GUI needs to manage environment variables too after all)? In other words, is a documented and supported Bitbucket Pipelines API in the pipeline already? wink

2 votes

As already mentioned by Jeroen (+1), the required Amazon ECR credentials are valid for 12 hours only, see Registry Authentication for details:

[...] you must authenticate your Docker client another way so that Amazon ECR knows who is requesting to push or pull an image. If you are using the Docker CLI, then use the docker login command to authenticate to an Amazon ECR registry with an authorization token that is provided by Amazon ECR and is valid for 12 hours. The GetAuthorizationToken API operation provides a base64-encoded authorization token that contains a user name (AWS) and a password that you can decode and use in a docker login command. 

Workaround for Bamboo

We have been facing a similar limitation when using the built-in Bamboo Docker task to push images to an ECR repository for subsequent use in the Amazon ECS tasks of our Tasks for AWS (Bamboo) add-on. In order to work around it, we have implemented an Amazon ECR Credentials Variables task, which provides the following variables for use with the Docker task (see How to push a Docker image to a repository in your Amazon ECR registry with the Bamboo Docker task for a step-by-step guide):

${bamboo.custom.aws.ecr.proxyEndpointDomain}
${bamboo.custom.aws.ecr.username}
${bamboo.custom.aws.ecr.password}

Partial workaround for Bitbucket Pipelines

Conceptually you should be able to implement a similar approach with Bitbucket Pipelines based on section Images hosted outside Docker Hub within Use Docker images as build environments in Bitbucket Pipelines, i.e. you must provide the registry details, for example:

image:
  name: $ECRPROXYENDPOINTDOMAIN/namespace/java:8u66
  username: $USERNAME
  password: $PASSWORD
  email: $EMAIL

That is, you simply provide the required credentials as Environment variables in Bitbucket Pipelines (note that $EMAIL can be arbitrary as it is ignored by Amazon ECR).

The final piece for rendering this a halfway decent workaround would be an API to update those environment variables programmatically from a scheduled external task (e.g. via a AWS Lambda function triggered by a scheduled CloudWatch event).

Unfortunately, and much to my surprise, Bitbucket Pipelines doesn't seem to offer any API yet, despite one of its main value propositions rightfully being 'configuration as code'. Thorough 'configuration as code' would also need to cover all aspects of Bitbucket Pipelines itself though, most importantly those environment variables for service discovery and credentials management etc. - a resp. feature request seems to be in order ...

Thanks for this extensive answer @Steffen Opel [Utoolity]! There is indeed a REST endpoint available to update a variable (so you can use it to update the ECR credentials): https://developer.atlassian.com/bitbucket/api/2/reference/resource/teams/%7Busername%7D/pipelines_config/variables/%7Bvariable_uuid%7D 

0 votes

Late answer but might help those googler's

You can do the below and it will log you in using your AWS access & secret keys in one line.

eval $(aws ecr get-login --no-include-email --region eu-west-2)

0 votes

There is now an easier way to do this!

See Images hosted by the AWS ECR in the docs

Suggest an answer

Log in or Join to answer
Community showcase
Piotr Plewa
Published Dec 27, 2017 in Bitbucket

Recipe: Deploying AWS Lambda functions with Bitbucket Pipelines

Bitbucket Pipelines helps me manage and automate a number of serverless deployments to AWS Lambda and this is how I do it. I'm building Node.js Lambda functions using node-lambda&nbsp...

643 views 0 4
Read article

Atlassian User Groups

Connect with like-minded Atlassian users at free events near you!

Find a group

Connect with like-minded Atlassian users at free events near you!

Find my local user group

Unfortunately there are no AUG chapters near you at the moment.

Start an AUG

You're one step closer to meeting fellow Atlassian users at your local meet up. Learn more about AUGs

Groups near you
Atlassian Team Tour

Join us on the Team Tour

We're bringing product updates and pro tips on teamwork to ten cities around the world.

Save your spot