Using Docker in Docker in a pipe

Question

I'm trying to create a pipe for customers that builds software for a niche platform, but I'm running into an issue.

One of the inputs for the pipe is the version of the software they're using. That translates to a Docker image tag that we have hosted on GitHub Container Registry (GHCR.io). So, at runtime, I need the pipe to read the version that the user has requested in the inputs, pull down a Docker image specific to that version, and then run it.

The problem I'm running into is that it seems that BitBucket doesn't allow Docker-in-Docker, so my pipe is unable to pull down or run the image. I've tried mounting /var/run/docker.sock when running my unit tests, I've tried running it as privileged, I've tried the DOCKER_BUILDKIT environment variable (both 1 and 0) and I get errors such as:

"Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"

"Error response from daemon: authorization denied by plugin pipelines: --privileged=true is not allowed."

"dial unix /var/run/docker.sock: connect: permission denied"

Has anyone built a custom pipe for BitBucket Pipelines that dynamically pulls down Docker images based on inputs to the pipe? What am I missing here?

Igor Stoyanov · Answer

@Paul Bromwell  hi. Maybe some additional info. The variable BITBUCKET_DOCKER_HOST_INTERNAL is available by default (also , i think DOCKER_HOST too). Below is a shell example how pipe docker container started to run with add-host parameters:
internal host example in pipeline -> --add-host="host.docker.internal:$BITBUCKET_DOCKER_HOST_INTERNAL", 
docker-host-env example in pipeline -> --env=DOCKER_HOST="tcp://host.docker.internal:2375"

If you are using python then try this inside your pipe:

# first install docker via pip
import os

import docker

docker_client = docker.from_env()

docker_client.containers.run(
'your-image',
    environment={
"DOCKER_HOST": "tcp://host.docker.internal:2375"  # it should be available by default, but you could explicitly set this here
    },
    extra_hosts={"host.docker.internal": os.getenv('BITBUCKET_DOCKER_HOST_INTERNAL')}
)

If you use bash then try this inside your pipe:

docker run ... --env=DOCKER_HOST="tcp://host.docker.internal:2375" \
--add-host="host.docker.internal:$BITBUCKET_DOCKER_HOST_INTERNAL" \ ...

Regards, Igor

Paul Bromwell · Answer

Hey Patrik!

No dice, I'm afraid. I still get this error even when adding to the /etc/hosts file (which is what extra_hosts is doing):

dial tcp: lookup host.docker.internal on 10.5.0.2:53: no such host

I did notice that if I have my Docker image run a shell script and run printenv (omitting the values I set), this is what it comes up with:

HOSTNAME=bf93cfeae826
SHLVL=1
HOME=/root
DIND_COMMIT=d58df1fc6c866447ce2cd129af10e5b507705624
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
DOCKER_VERSION=24.0.6
DOCKER_TLS_CERTDIR=/certs
DOCKER_HOST=tcp://host.docker.internal:2375
DOCKER_BUILDX_VERSION=0.11.2
DOCKER_COMPOSE_VERSION=2.23.0
PWD=/

So I don't think that BITBUCKET_DOCKER_HOST_INTERNAL environment variable is getting passed to the containers.

I'm just going to rework the code to not have to use Docker-in-Docker.

Thanks anyway!

Paul

Patrik S · Answer

Hello @Paul Bromwell , It seems like additionally to ""DOCKER_HOST": "tcp://host.docker.internal:2375 you should also provide: extra_hosts={"host.docker.internal": os.getenv('BITBUCKET_DOCKER_HOST_INTERNAL')} as in the following example of one of the Atlassian-developed pipes: * http://atlassian/aws-ecr-push - Docker in Docker and referenced on docker-py documentation : * https://docker-py.readthedocs.io/en/stable/containers.html Thank you! Patrik S