Use Docker within Bitbucket pipe

[christoph]

Hi,

I wrote my 1st own Bitbucket Pipe using Python. It's supposed to tag a docker image and push it to a registry. So I have to use Docker within the pipe itself. Everything is working nicely locally. But using my pipe in a Bitbucket pipeline fails because I don't get Docker to work. Here is what I try to do:

def get_client(self):
      if self.docker_client is None:
        self.docker_client = docker.APIClient(base_url='unix://var/run/docker.sock')
      
      return self.docker_client

The complete code of my pipe can be found in this public repository: https://bitbucket.org/humanitec/push-to-humanitec/src/master/

I cannot find any documentation on how to connect to Docker from within a Bitbucket Pipe. Does anybody have experience with that?

Thanks and cheers,

Christoph

[ktomk]

The docker daemon is not needed within the pipe container, just the docker client.

When the pipe is started, the docker socket will be mounted into the pipe container, the docker client then can do it's job. 

[christoph]

Thanks. But how do I call the client? from_env() does not seem to work for me.

[ktomk]

@christoph were you able to install the binary of the docker client? the "docker" command should then be available in the container. e.g. "docker --version" should give some output.

[christoph]

Yes @ktomk. A "docker --version" run inside the container leads to the following output "INFO: docker --version = 3.7.0".

But the command "docker_client = docker.from_env()" fails.

I also tried "docker_client = docker.APIClient(base_url='unix://var/run/docker.sock')" since I need the low-level APi. But this does not seem to work.

[ktomk]

I assume you use some software runtime here (python?) and that is about specific libraries in there. I'm not a strong practitioner at python, most likely the normally fine command "docker.from_env()" does not work out of the box within the pipeline / pipe environment? (at least for you, no pun intended)

This can lead to a lot of guess work. It might be more straight forward to check the specification of the lib and if the example container image you mentioned earlier is similar, to review the sources of it what might be different and more importantly what makes the difference. Depending on your way of work the first or the second works better for you.

As (let's say) it's python, fast feedback cycles might be possible by having a build script to build the pipe container and then call it with the parameters on your local system by executing the docker command. When you setup a pipeline in a bitbucket repository and run it, bitbucket shows how the pipe is executed. this can be easily mocked locally with much faster iteration.

Keeping the pipe container running after fail does also allow to inspect it. Something that is not feasible within a pipeline run on bitbucket cloud.

I hope that helps a bit. I do see in the repo you linked that it's the same command in python, so don't want to sound ignorant here. just saying.

[benoitscherrer]

Hi Christoph,

Have you been successful in making ```docker_client = docker.from_env()``` work in a bitbucket pipeline?

I'm running into the same problem...

Thanks!
B

[christoph]

Hi Benoit,

no... I have not. I am now using Kaniko within the pipeline to get the job done.

Cheers,

Christoph

[benoitscherrer]

Thanks for the reply! Actually I realized the docker service was in other steps but not in this one (my bad).

When adding it, with something like:

---------
   - step:
       script:
         - pip install poetry==1.1.4
            [...]
         - poetry install -vv
         - poetry run pytest -v --tb=line

       services:
         - docker
       caches:
         - docker
         - pip
         - poetry-path
         - poetry-venv

-----------

then docker_client=docker.from_env()  worked for me

[christoph]

Good point... that's a different case.