Pipelines access docker host

[crash21]

Hello,

I have set the runners, self hosted to my server. Everything works fine, runner is working and spins a new container which can also have docker using docker service.

pipelines:
    branches:
        master:
         - step:
            services:
              - docker
            runs-on:
              - self.hosted
              - linux
            script:
              - docker ps -a
              - docker images
              - echo "OK"

 Now my problem is that docker ps -a and docker images are empty.

I think docker in docker works but is completely isolated from the host docker. However, this does not fulfill my need.

In the host (the same host the runner is running), I have the container that needs to be updated from the pipeline. I want to build a new image and run then: docker compose -f xxx.yml up -d

How can I do this using docker runner? Is the only solution using Linux shell instead which i will install directly on the host?

[Theodora Boudale · Accepted solution]

Hi @crash21,

This is not possible with Linux Docker Runners, as they limit access to the host.

One thing you could do is commit the Dockerfile and any other files you need to build the Docker image in your repo. These files will then be available during the build in the clone directory, so you can build the Docker image, start a container, push the image to a registry etc.

Otherwise, you can use a Linux Shell Runner, which provides access to the host. Please keep in mind that if you do things such as installing any applications, starting a database service, or editing a file outside of the build directory, during a build, these would potentially affect the next step to be run. So, you'll need to make sure that the scripts you run in each step won’t have a major impact on other steps or builds.

If you use a Linux Shell Runner, you will need to add the following command at the beginning of your script or before you run any Docker commands, in order to be able to use Docker:

- export DOCKER_HOST=""

This is because we are setting the variable DOCKER_HOST to tcp://localhost:2375, which can lead to issues.

If you have any questions, please feel free to let me know.

Kind regards,
Theodora