How to use a Docker image from private registry for an isolated build inside a

[Nikolay]

In general I'm trying to implement this guide here: https://confluence.atlassian.com/bamboo/docker-runner-946020207.html

The one point which is still not clear for me is how to use an Docker image from my private Docker registry. Actually where to add the credentials used for the Docker registry?

Currently, the build fails with the following error message:

29-Sep-2021 15:43:55 Unable to find image 'private-registry/image_name:tag' locally

29-Sep-2021 15:43:55 docker: Error response from daemon: Get https://private-registry/v2/image_name/manifests/tag: unknown: Authentication is required.

29-Sep-2021 15:43:55 See 'docker run --help'.

29-Sep-2021 15:43:55 Exit code: 125, output: STDOUT: Beginning to execute external process for build 'Test build in a Docker container - Default Job #11 (TBIDC-JOB1-11)'\n ... running command line: \n/usr/bin/docker run --volume /opt/atlassian/bamboo/bamboo-agent-home/xml-data/build-dir/TBIDC-JOB1:/opt/atlassian/bamboo/bamboo-agent-home/xml-data/build-dir/TBIDC-JOB1 --volume /opt/atlassian/bamboo/bamboo-agent-home/temp:/opt/atlassian/bamboo/bamboo-agent-home/temp --detach --name d7b55496-b441-43c9-bc3a-c219fae242ae34276881 private-registry/image_name:tag tail -f /dev/null\n ... in: /opt/atlassian/bamboo/bamboo-agent-home/xml-data/build-dir/TBIDC-JOB1\nSTDERR: Unable to find image 'private-registry/image_name:tag' locallySTDERR: docker: Error response from daemon: Get https://private-registry/v2/image_name/manifests/tag: unknown: Authentication is required.STDERR: See 'docker run --help'.

29-Sep-2021 15:43:55 Failed to start docker container d7b55496-b441-43c9-bc3a-c219fae242ae34276881

29-Sep-2021 15:43:55 com.atlassian.bamboo.docker.DockerException: Error running Docker run command

29-Sep-2021 15:43:55 at com.atlassian.bamboo.docker.DockerContainerServiceImpl.run(DockerContainerServiceImpl.java:134)

29-Sep-2021 15:43:55 at com.atlassian.bamboo.v2.build.agent.docker.DockerContainerControllerImpl.startContainerIfRequired(DockerContainerControllerImpl.java:103)

29-Sep-2021 15:43:55 at com.atlassian.bamboo.v2.build.agent.DefaultBuildAgent.build(DefaultBuildAgent.java:160)

29-Sep-2021 15:43:55 at com.atlassian.bamboo.v2.build.agent.BuildAgentControllerImpl.lambda$waitAndPerformBuild$0(BuildAgentControllerImpl.java:131)

29-Sep-2021 15:43:55 at com.atlassian.bamboo.variable.CustomVariableContextImpl.withVariableSubstitutor(CustomVariableContextImpl.java:118)

29-Sep-2021 15:43:55 at com.atlassian.bamboo.v2.build.agent.BuildAgentControllerImpl.waitAndPerformBuild(BuildAgentControllerImpl.java:125)

29-Sep-2021 15:43:55 at com.atlassian.bamboo.v2.build.agent.DefaultBuildAgent.lambda$start$0(DefaultBuildAgent.java:110)

29-Sep-2021 15:43:55 at com.atlassian.bamboo.utils.BambooRunnables$1.run(BambooRunnables.java:48)

29-Sep-2021 15:43:55 at com.atlassian.bamboo.security.ImpersonationHelper.runWith(ImpersonationHelper.java:26)

29-Sep-2021 15:43:55 at com.atlassian.bamboo.security.ImpersonationHelper.runWithSystemAuthority(ImpersonationHelper.java:17)

29-Sep-2021 15:43:55 at com.atlassian.bamboo.security.ImpersonationHelper$1.run(ImpersonationHelper.java:41)

29-Sep-2021 15:43:55 at java.lang.Thread.run(Thread.java:748)

29-Sep-2021 15:43:55 Caused by: com.atlassian.utils.process.ProcessException: Error executing /usr/bin/docker run --volume /opt/atlassian/bamboo/bamboo-agent-home/xml-data/build-dir/TBIDC-JOB1:/opt/atlassian/bamboo/bamboo-agent-home/xml-data/build-dir/TBIDC-JOB1 --volume /opt/atlassian/bamboo/bamboo-agent-home/temp:/opt/atlassian/bamboo/bamboo-agent-home/temp --detach --name d7b55496-b441-43c9-bc3a-c219fae242ae34276881 private-registry/image_name:tag tail -f /dev/null

29-Sep-2021 15:43:55 at com.atlassian.bamboo.docker.DockerContainerServiceImpl.execute(DockerContainerServiceImpl.java:306)

29-Sep-2021 15:43:55 at com.atlassian.bamboo.docker.DockerContainerServiceImpl.execute(DockerContainerServiceImpl.java:277)

29-Sep-2021 15:43:55 at com.atlassian.bamboo.docker.DockerContainerServiceImpl.run(DockerContainerServiceImpl.java:111)

29-Sep-2021 15:43:55 ... 11 more

29-Sep-2021 15:43:55 

Beginning to execute external process for build 'Test build in a Docker container - Default Job #11 (TBIDC-JOB1-11)'

 ... running command line: 

/usr/bin/docker rm -f d7b55496-b441-43c9-bc3a-c219fae242ae34276881

 ... in: /opt/atlassian/bamboo/bamboo-agent-home/xml-data/build-dir/TBIDC-JOB1

29-Sep-2021 15:43:55 Error: No such container: d7b55496-b441-43c9-bc3a-c219fae242ae34276881

29-Sep-2021 15:43:55 Exit code: 1, output: STDOUT: Beginning to execute external process for build 'Test build in a Docker container - Default Job #11 (TBIDC-JOB1-11)'\n ... running command line: \n/usr/bin/docker rm -f d7b55496-b441-43c9-bc3a-c219fae242ae34276881\n ... in: /opt/atlassian/bamboo/bamboo-agent-home/xml-data/build-dir/TBIDC-JOB1\nSTDERR: Error: No such container: d7b55496-b441-43c9-bc3a-c219fae242ae34276881

29-Sep-2021 15:43:55 Failed to remove docker container d7b55496-b441-43c9-bc3a-c219fae242ae34276881

29-Sep-2021 15:43:55 com.atlassian.bamboo.docker.DockerException: Error running Docker remove command

29-Sep-2021 15:43:55 at com.atlassian.bamboo.docker.DockerContainerServiceImpl.remove(DockerContainerServiceImpl.java:267)

29-Sep-2021 15:43:55 at com.atlassian.bamboo.v2.build.agent.docker.DockerContainerControllerImpl.removeContainer(DockerContainerControllerImpl.java:152)

29-Sep-2021 15:43:55 at com.atlassian.bamboo.v2.build.agent.docker.DockerContainerControllerImpl.startContainerIfRequired(DockerContainerControllerImpl.java:111)

29-Sep-2021 15:43:55 at com.atlassian.bamboo.v2.build.agent.DefaultBuildAgent.build(DefaultBuildAgent.java:160)

29-Sep-2021 15:43:55 at com.atlassian.bamboo.v2.build.agent.BuildAgentControllerImpl.lambda$waitAndPerformBuild$0(BuildAgentControllerImpl.java:131)

29-Sep-2021 15:43:55 at com.atlassian.bamboo.variable.CustomVariableContextImpl.withVariableSubstitutor(CustomVariableContextImpl.java:118)

29-Sep-2021 15:43:55 at com.atlassian.bamboo.v2.build.agent.BuildAgentControllerImpl.waitAndPerformBuild(BuildAgentControllerImpl.java:125)

29-Sep-2021 15:43:55 at com.atlassian.bamboo.v2.build.agent.DefaultBuildAgent.lambda$start$0(DefaultBuildAgent.java:110)

29-Sep-2021 15:43:55 at com.atlassian.bamboo.utils.BambooRunnables$1.run(BambooRunnables.java:48)

29-Sep-2021 15:43:55 at com.atlassian.bamboo.security.ImpersonationHelper.runWith(ImpersonationHelper.java:26)

29-Sep-2021 15:43:55 at com.atlassian.bamboo.security.ImpersonationHelper.runWithSystemAuthority(ImpersonationHelper.java:17)

29-Sep-2021 15:43:55 at com.atlassian.bamboo.security.ImpersonationHelper$1.run(ImpersonationHelper.java:41)

29-Sep-2021 15:43:55 at java.lang.Thread.run(Thread.java:748)

29-Sep-2021 15:43:55 Caused by: com.atlassian.utils.process.ProcessException: Error executing /usr/bin/docker rm -f d7b55496-b441-43c9-bc3a-c219fae242ae34276881

29-Sep-2021 15:43:55 at com.atlassian.bamboo.docker.DockerContainerServiceImpl.execute(DockerContainerServiceImpl.java:306)

29-Sep-2021 15:43:55 at com.atlassian.bamboo.docker.DockerContainerServiceImpl.execute(DockerContainerServiceImpl.java:277)

29-Sep-2021 15:43:55 at com.atlassian.bamboo.docker.DockerContainerServiceImpl.remove(DockerContainerServiceImpl.java:265)

29-Sep-2021 15:43:55 ... 12 more

What I have already tried:

I created the ~/.dockercfg file as described here: Configuring the Docker task in Bamboo under Advanced authentication.

But to be honest I'm not sure if this is also working for a pull of a Docker image or just in case the image has to be pushed.

What I don't want do use is to add the user credentials inside the ~/.docker/config.json file.

Unfortunately I spend already many hours searching for a solution but without any success. I hope anyone can help me and is using Bamboo in with a similar configuration.

Bamboo version: 7.1.4

Docker version: 19.03.5

BTW. using an Docker image from Dockerhub is working!

[Steffen Opel _Utoolity_]

Hi @Nikolay, welcome to the Atlassian Community!

I'm afraid the capability your are looking for is not yet available, please watch and vote for the following issue to increase Atlassian's priority for addressing this notable limitation:

• Add Capability for 'Docker Runner' to connect to private image registry (BAM-19731) 

Potential workaround

I guess the workaround via docker login mentioned in the issue description is too similar to the one via ~/.docker/config.json that you already excluded, but thanks to your question I  just noticed a recently added workaround by Aron Rosenberg that might work for some scenarios:

1. Create a two stage build. Call the 1st stage "Fetch the Container", and the 2nd stage "Build"
2. Configure the 1st Stage to use an Agent Environment and have one job called "Fetch".
   1. Add a Docker task to the "Fetch" job.
      1. Set the Command to "Pull from a repository".
      2. Select custom registry and put your container URL in here including the version tag (You can use Bamboo variables if you want plan branches to pull different tags)
      3. Set the Authentication Type to your needed requirements (Shared credentials works well)
3. Configure the 2nd Stage to use Docker as the runner
   1. Put the exact same container URL used in the 1st stage Docker Pull task as the "Docker Image" value.
4. Add a new unique Requirement to the 1st and 2nd stage and make sure your agent has the same Requirement set on it. Using a Custom Requirement works great for this. This will force the same agent to be reused on each stage.

As Aron explains, "This works because Docker pull just fetches the image into the machine local cache [so that the 2nd stage] doesn't need additional authentication".

Good luck,
Steffen