It's not the same without you

Join the community to find out what other Atlassian users are discussing, debating and creating.

Atlassian Community Hero Image Collage

Pipelines: Pulling docker images from a Amazon ECR repository

For a build pipeline I would like to start from a docker image stored in a Amazon EC2 Container Registry (ECR) repository. Pulling this image would normally require me to do a 'aws ecr get-login' first, but I fail to see how this would work with a in a bitbucket-pipelines.yml file. How do I get this working,? Since the pull is the first step of the build pipeline, I don't think I can execute a script?



4 answers

1 accepted

1 vote
Answer accepted
davina Atlassian Team Oct 29, 2017

There is now an easier way to do this!

See Images hosted by the AWS ECR in the docs

2 votes

It looks like Amazon ECR currently only allows you to generate Docker credentials that are valid for 12 hours:

Generating these for your build should work, however I suppose it is quite inconvenient to regenerate them all of the time. 

I've just outlined a potential workaround, only to stumble over the apparent lack of an actual Bitbucket Pipelines API - presumably this is just not officially documented yet (the GUI needs to manage environment variables too after all)? In other words, is a documented and supported Bitbucket Pipelines API in the pipeline already? wink

2 votes

As already mentioned by Jeroen (+1), the required Amazon ECR credentials are valid for 12 hours only, see Registry Authentication for details:

[...] you must authenticate your Docker client another way so that Amazon ECR knows who is requesting to push or pull an image. If you are using the Docker CLI, then use the docker login command to authenticate to an Amazon ECR registry with an authorization token that is provided by Amazon ECR and is valid for 12 hours. The GetAuthorizationToken API operation provides a base64-encoded authorization token that contains a user name (AWS) and a password that you can decode and use in a docker login command. 

Workaround for Bamboo

We have been facing a similar limitation when using the built-in Bamboo Docker task to push images to an ECR repository for subsequent use in the Amazon ECS tasks of our Tasks for AWS (Bamboo) add-on. In order to work around it, we have implemented an Amazon ECR Credentials Variables task, which provides the following variables for use with the Docker task (see How to push a Docker image to a repository in your Amazon ECR registry with the Bamboo Docker task for a step-by-step guide):


Partial workaround for Bitbucket Pipelines

Conceptually you should be able to implement a similar approach with Bitbucket Pipelines based on section Images hosted outside Docker Hub within Use Docker images as build environments in Bitbucket Pipelines, i.e. you must provide the registry details, for example:

  name: $ECRPROXYENDPOINTDOMAIN/namespace/java:8u66
  username: $USERNAME
  password: $PASSWORD
  email: $EMAIL

That is, you simply provide the required credentials as Environment variables in Bitbucket Pipelines (note that $EMAIL can be arbitrary as it is ignored by Amazon ECR).

The final piece for rendering this a halfway decent workaround would be an API to update those environment variables programmatically from a scheduled external task (e.g. via a AWS Lambda function triggered by a scheduled CloudWatch event).

Unfortunately, and much to my surprise, Bitbucket Pipelines doesn't seem to offer any API yet, despite one of its main value propositions rightfully being 'configuration as code'. Thorough 'configuration as code' would also need to cover all aspects of Bitbucket Pipelines itself though, most importantly those environment variables for service discovery and credentials management etc. - a resp. feature request seems to be in order ...

Thanks for this extensive answer @Steffen Opel [Utoolity]! There is indeed a REST endpoint available to update a variable (so you can use it to update the ECR credentials): 

Late answer but might help those googler's

You can do the below and it will log you in using your AWS access & secret keys in one line.

eval $(aws ecr get-login --no-include-email --region eu-west-2)

Suggest an answer

Log in or Sign up to answer
Community showcase
Published in Bitbucket

Atlassian supported Jenkins integration for Bitbucket Server

We’ve been building a plugin to integrate Bitbucket Server and Jenkins CI, and I’m excited to announce that our alpha is ready to download and install. It lets you seamlessly configure a Jenkins job ...

1,087 views 3 17
Read article

Community Events

Connect with like-minded Atlassian users at free events near you!

Find an event

Connect with like-minded Atlassian users at free events near you!

Unfortunately there are no Community Events near you at the moment.

Host an event

You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events

Events near you