As a part of the ongoing security hardening process in Bitbucket Pipelines, we decided to enable docker userns remap to eliminate potential security risks that might affect our users.
This change affected a small percentage of people using docker in their builds. So, we wanted to share the 4 different limitations that this change introduced, and the actions required to make your builds work in the more secure system.
As mentioned in the docker documentation linked above, enabling docker userns remap introduces the following limitations:
The following standard Docker features are incompatible with running a Docker daemon with user namespaces enabled:
- sharing PID or NET namespaces with the host (--pid=host or --network=host).
- external (volume or storage) drivers which are unaware or incapable of using daemon user mappings.
- Using the --privileged mode flag on docker run without also specifying --userns=host.
In our previous security updates we disallowed pid=host and --privileged containers however network=host is a new limitation.
As network=host will no longer be allowed, there are a few other ways you can communicate between your code and containers.
To still allow your containers and code to communicate please see the following guidance.
If you need to communicate with a service running in docker from your build container, when starting the service provide it a port mapping using -p <host-port>:<container-port> you can then access the service using localhost:<host-port>.
If you need to communicate from a service running in docker to a service running in your build container, when starting the service provide it the following host entry using --add-host host.docker.internal:$BITBUCKET_DOCKER_HOST_INTERNAL you can then access the service using host.docker.internal:<port>.
If you need to talk between services running in docker, they have to be attached to the same docker network so you can address the services using <container-name>:<container-port> or consider using docker compose to run the services which creates a custom network for the compose stack.
Due to the way userns remap works (by remapping UID/GIDs to another less privileged UID/GID in the hosts namespace), UID/GIDs that are placed on files must be in the range 0-65535.
If you recieve this error, you will need to perform a fix on the image that has this invalid UID/GID.
If your recieved this error against a file that you created and placed into an image, the fix is to change the UID/GID of the file to one within the range.
To find the file, first get the invalid UID/GID from the error message and run the following command on the files within the container.
find / -uid|-gid <invalid-uid-or-guid> -ls
To fix the ownership of the file use the following command
chown -R <uid-in-range>:<gid-in-range> filename
For a multi-stage build, you may need to modify the command above slightly as shown below
chown -R root:root filename
If you received this error against a file that was in the base image you depend on, you will need to raise a support case and work with the image maintainers of the base image to get this resolved.
Test containers recently released a fix to disable ryuk in version 1.10.6 by setting the environment variable TESTCONTAINERS_RYUK_DISABLED to true as documented here. In order to keep using test containers in pipelines you will need to upgrade your test containers dependency to this version or greater and set the environment variable in your pipelines (either directly in the step, or via an account/repository variable).
Due to bugs in docker, when files were modified (removed/alterered) during a docker build via RUN/COPY commands the changes weren't being reflected in subsequent layers.
We have upgraded docker to pull in the fix for these issues.
However some failures require user intervention, as some commands were modifying the UID/GID (similar to the error below). For these situations docker now supports --chown as an argument to some dockerfile commands.
Hey Community! I work on the Bitbucket product marketing team. With Halloween approaching, we wanted to discuss a topic tailor-made for October: development horror stories. Whether it was a lurk...
Connect with like-minded Atlassian users at free events near you!Find an event
Connect with like-minded Atlassian users at free events near you!
Unfortunately there are no Community Events near you at the moment.Host an event
You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events