How to push docker image tagged after git branch?

[Sergey Mokhov]

Hi everyone!

We are in the middle of transition to bitbucket pipelines.

We build docker images from every git branch and run tests against these images. To make tracking of what image was build from what branch easier we tag docker images with git branch name it was built from.

The problem is that feature branches have slash "/' in their name and slash is not allowed in docker image tag. What we used to do is replace slashes with dashes and use that as a tag.

One option for us would be to use a script that does that substitution and pushes image, but we would have to put that script in every repository.

To use pipe atlassian/aws-ecr-push-image:1.2.2 we tried to define tags as

TAGS: '${BITBUCKET_BRANCH//\//-}'
or
TAGS: ${BITBUCKET_BRANCH//\//-}

but it does not work and build fails with

Image not found: 404 Client Error: Not Found 

Image not found: 404 Client Error: Not Found ("no such image: whatever: No such image: whatever:latest")

 We also tried to add a separate step that does replacement and exports result as  another variable, but that did not work either.

Changing naming convention would cause a lot of pain, many scripts rely on existing scripts rely on that naming convention.

[Halyna Berezovska · Accepted solution]

@Sergey Mokhov I guess you want something like changing,e.g

feature/jira-story-333 to feature-jira-story-333.

You can define another local variable to substisute, BUT with proper escaping before executing pipe, like

script:

 - deploy_tag="${BITBUCKET_BRANCH/\//-}"

- pipe: atlassian/aws-ecr-push-image:1.2.0

  variables:

      ....

      TAGS: $deploy_tag

  .....

for me replacing worked all right:

BITBUCKET_BRANCH=feature/check-branch

branch="${BITBUCKET_BRANCH/\//-}"
echo "branch is" $branch
branch is feature-check-branch

Perhaps, passing such substitution right in the variables does not work, but try to initialize this variable and let's see. In the pipe we accept this variable as string and I am not sure it is acceptable to pass value as a variable and it will be transformed on the fly. So then try just with local var initializing.

Also, please share logs after the execution if it fails.

Regards, Galyna

[Sergey Mokhov]

Hi Galyna, thank you for your reply. Unfortunately it did not work.

- step: &build-deploy-pes-image
 name: Build and push PES Docker image
caches:
 - gradle
- gradlewrapper
- docker
services:
 - docker
script:
 - scripts/build-docker-image.sh "${BITBUCKET_BRANCH}"
- deploy_tag="${BITBUCKET_BRANCH/\//-}"
- echo "Deploy tag is ${deploy_tag}"
- pipe: atlassian/aws-ecr-push-image:1.2.2
variables:
 AWS_ACCESS_KEY_ID: "${KEY}"
 AWS_SECRET_ACCESS_KEY: "${SECRET}"
 IMAGE_NAME: xxx.amazonaws.com/pfpes-web
TAGS: "${deploy_tag}"

Pipe image was called with

--env=TAGS="${deploy_tag}" 

 And failed like that

Image not found: 404 Client Error: Not Found ("no such image: xxx.amazonaws.com/pfpes-web: No such image: xxx.amazonaws.com/pfpes-web:latest")

[Halyna Berezovska]

@Sergey Mokhov 

I just jave tried our pipe.

I am not sure how you setup your infra in AWS ECR, so just in case ensure that you take into account following details, that I did and it worked for me (firstly it also was failing with the same error you provided):

- your IMAGE_NAME should equal to repository name you created in AWS ECR

- when you go to your repository in AWS console and click push commands, it explains you what to do before push, the main thing is:

docker build -t <your_repository_name> .

So ensure you build that same image you're trying to push.

Everything else is our pipe can take care of, I suppose (for me it worked).

So my script is:

- step:
    name: test ecr
    script:
      - echo "Test ecr"
      - BRANCH="${BITBUCKET_BRANCH/\//-}"
      - echo "branch is" $BRANCH
      - docker build -t bbci-pipes-test-ecr-239 .
      - pipe: atlassian/aws-ecr-push-image:1.2.2
        variables:
          AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID
          AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY
          AWS_DEFAULT_REGION: $AWS_DEFAULT_REGION
          IMAGE_NAME: 'bbci-pipes-test-ecr-239'
          TAGS: ${BRANCH}

And it worked. Screenshot is attached.

If it does not work, share more details about your setup and share full trace just in case, not just error output. This will help to debug your case.

Regards, Galyna

[Sergey Mokhov]

I do not know why it does not show my previous comments, but...

I tried image name with no aws url and it did not work either.

Image not found: 404 Client Error: Not Found ("no such image: pfpes-web: No such image: pfpes-web:latest")

[Halyna Berezovska]

@Sergey Mokhov

check the details I pointed to :

- if you name the image pfpes-web, the repository, that contains your images in aws should  be named pfpes-web.

- how you build your docker image in your scripts/build-docker-image.sh script?

These details matters, I have tried that and had the same error and then I fixed this with

* building image script fix

* naming image as the same that my remote repository.

Please, share with us these details about your setup in aws console and we can help to fix it, I don't see you provided them

[Sergey Mokhov]

Repository name matches image name exactly and it is pfpes-web.

The script builds image as:

finalImageName="$dockerRepositoryName/pfpes-web:$branchName"
docker build -t "${finalImageName}" .

 So the resulting image name looks like xxxxxxxxxx.dkr.ecr.region-whatever.amazonaws.com/pfpes-web:feature-whatever

I can push this image into ECR with

docker push "${dockerRegistryAddress}/${ecrRepoName}:${branchName}"

[Halyna Berezovska]

@Sergey Mokhov try to go with

finalImageName=pfpes-web // name of your repository
docker build -t ${finalImageName}

 THe point is , we are getting registry address in our pipe ourselves. We think it should make things simpler:

1. By your credentials pipe gets token from aws

2. Using token pipe discovers your registry and creds ITSELF.

So you don't need to type it in.

If that solution is not fine for you (when we discover your address ourselves by your creds inside the pipe and it is more safe in our opinion), within a team we can think about feature request to the pipe, sth like custom registry. In this case we would need from you more details to decide things about the implementation of such custom registry address.

But IMHO, if you have an ability to not show up such things in the pipeline, you can use this ability.

This is up to you, actually, if it is necessary for you to go with registry address, we can talk about this .

Waiting for your feedback!

Cheers, Galyna

[Sergey Mokhov]

Thank you Galina for your answers. I have not tried the last one you suggested, but pretty sure it would work.