It's not the same without you

Join the community to find out what other Atlassian users are discussing, debating and creating.

Atlassian Community Hero Image Collage

Changes to make your containers more secure on Bitbucket Pipelines.

As a part of the ongoing security hardening process in Bitbucket Pipelines, we decided to enable docker userns remap to eliminate potential security risks that might affect our users.

This change affected a small percentage of people using docker in their builds. So, we wanted to share the 4 different limitations that this change introduced, and the actions required to make your builds work in the more secure system.

Issues caused by limitations of userns remap

1. Docker: network=host no longer supported

As mentioned in the docker documentation linked above, enabling docker userns remap introduces the following limitations:

The following standard Docker features are incompatible with running a Docker daemon with user namespaces enabled:

- sharing PID or NET namespaces with the host (--pid=host or --network=host).

- external (volume or storage) drivers which are unaware or incapable of using daemon user mappings.

- Using the --privileged mode flag on docker run without also specifying --userns=host.

In our previous security updates we disallowed pid=host and --privileged containers however network=host is a new limitation.

As network=host will no longer be allowed, there are a few other ways you can communicate between your code and containers.

To still allow your containers and code to communicate please see the following guidance.

If you need to communicate with a service running in docker from your build container, when starting the service provide it a port mapping using -p <host-port>:<container-port> you can then access the service using localhost:<host-port>.

If you need to communicate from a service running in docker to a service running in your build container, when starting the service provide it the following host entry using --add-host host.docker.internal:$BITBUCKET_DOCKER_HOST_INTERNAL you can then access the service using host.docker.internal:<port>.

If you need to talk between services running in docker, they have to be attached to the same docker network so you can address the services using <container-name>:<container-port> or consider using docker compose to run the services which creates a custom network for the compose stack.

2. Builds fails due to error: “Container ID Cannot Be Mapped to Host ID Error

Due to the way userns remap works (by remapping UID/GIDs to another less privileged UID/GID in the hosts namespace), UID/GIDs that are placed on files must be in the range 0-65535.

If you recieve this error, you will need to perform a fix on the image that has this invalid UID/GID.

If your recieved this error against a file that you created and placed into an image, the fix is to change the UID/GID of the file to one within the range.

To find the file, first get the invalid UID/GID from the error message and run the following command on the files within the container.

find / -uid|-gid <invalid-uid-or-guid> -ls

To fix the ownership of the file use the following command

chown -R <uid-in-range>:<gid-in-range> filename

For a multi-stage build, you may need to modify the command above slightly as shown below

chown -R root:root filename

If you received this error against a file that was in the base image you depend on, you will need to raise a support case and work with the image maintainers of the base image to get this resolved.

3. Test containers

Test containers recently released a fix to disable ryuk in version 1.10.6 by setting the environment variable TESTCONTAINERS_RYUK_DISABLED to true as documented here. In order to keep using test containers in pipelines you will need to upgrade your test containers dependency to this version or greater and set the environment variable in your pipelines (either directly in the step, or via an account/repository variable).

4. Docker build: removing files doesn’t remove them

Due to bugs in docker, when files were modified (removed/alterered) during a docker build via RUN/COPY commands the changes weren't being reflected in subsequent layers.

We have upgraded docker to pull in the fix for these issues.

However some failures require user intervention, as some commands were modifying the UID/GID (similar to the error below). For these situations docker now supports --chown as an argument to some dockerfile commands.


My build fails with error: Container ID 165586 cannot be mapped to a host ID

So, I guess I am falling into the second case. Although I tried your suggestion, I couldn't find any file with id `165586`. 


Any clue?

Ashvin Narayanan Mar 01, 2019

I'm having exactly the same issue @Panagiotis Matsinopoulos .

Error processing tar file(exit status 1): Container ID 165586 cannot be mapped to a host ID

Like likes this Mar 01, 2019 • edited

If you are using pip to install python packages, try using the --target flag to prevent pip from working in core OS file system spaces with lower privileges:


RUN mkdir -p /src
RUN pip install --no-cache-dir --target=/src -r requirements.txt

Panagiotis Matsinopoulos Mar 02, 2019 • edited Mar 03, 2019

What finally worked for me was the following:

I removed from my Dockerfile the update of the pip.

So, instead of:

RUN pip install --upgrade pip pipenv

I did:

Run pip install pipenv

and it worked. Maybe the version of the pip that I was trying to upgrade to had the problem with the file permissions, but didn't have time to look at it in more detail.

Like # people like this Mar 04, 2019

@Panagiotis Matsinopoulos Your solution is fair and makes sense to my hypothesis: if the base pip of the docker image is compliant with the user namespace mapping, then everything is fine; whereas upgrading in the new image rewrites the directories with underprivileged user outside the legal uid range for userns.  I haven't done a deep enough dive to prove this is absolutely the case, but if it is, your solution is vulnerable to:

1. Changes in the base image.  Docker tags do not imply immutability.  For instance, the official python image with tag 3.6 was updated 2 days ago.  

2. Never being able to update pip to a desired version without first modifying or raising an issue related to the base image.

The first problem can lead to another automagical failure of pipelines without warning.  I would consider this critical.  I don't anticipate docker image maintainers to prioritize bitbucket pipelines, so raising an issue will probably yield no results.


I consider both of our solutions workarounds.  But, I do not see a clean solution being provided by Bitbucket/moby, Docker, or pip anytime soon.  At least in the --target solution, minimal changes to the Dockerfile can provide stronger confidence that the pipeline is resilient to uncontrollable external factors. (Again assuming my hypothesis is correct)


Log in or Sign up to comment
Community showcase
Published in Bitbucket Pipelines

Building a Bitbucket Pipe as a casual coder :  #!/bin/bash source "$(dirname "$0")/" enable_debug extra_args="" if [[ "${DEBUG}" == "true" ]]; then extra_args="--verbose" fi # mandatory variables R...

1,652 views 1 18
Read article

Community Events

Connect with like-minded Atlassian users at free events near you!

Find an event

Connect with like-minded Atlassian users at free events near you!

Unfortunately there are no Community Events near you at the moment.

Host an event

You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events

Events near you