How to connect docker daemon from bitbucket pipelines step?

[ethembynkr]

We need to connect to docker where bitbucket runner is running. Is this possible?

[lassian]

Hi ethembynkr,

We don't currently allow users to access the hosts docker daemon, we do provide a docker in docker daemon as part of the step and in a future release are looking to allow users to override this with a dind image of their choosing (so they can run privileged containers and access host networking etc).

In the meantime, it would be interesting to know your use case for accessing the hosts docker daemon, if your willing to share :)

Kind Regards,

Nathan Burrell

[ethembynkr]

We want to create and update docker services in host docker through bitbucket-pipelines. (Server is not accessible from the outside.)

Like this

[lassian]

So essentially your using pipelines and the self hosted runner to update services on the actual nodes running them?

Interesting use case :)

cc @Justin Thomas 

Kind Regards,

Nathan Burrell

[mariyandimitrov]

Hey @lassian , related to your comment, more specifically this part "we do provide a docker in docker daemon as part of the step and in a future release are looking to allow users to override this with a dind image of their choosing" - my use case is that I want to run kind in a step to run tests against helm charts (my step is bellow), is this currently possible via some configuration in the runner (I see gitlab runners let you enable privileged for instance) or in the pipeline (perhaps using a different service than docker, referring to some specific image etc.)? In case it is something you plan on releasing later, could you maybe share the feature/story if there is such, so I could watch it? Thanks a lot and sorry if I'm hijacking the topic.

Thanks!

Mariyan

[mariyandimitrov]

The step in question (the image used is golang 1.16 on alpine):

 - step: &test_cls
name: CLS Tests
runs-on: self.hosted
script:
- cd $BITBUCKET_CLONE_DIR/tests/cls
- *get_deps
- make tools
- make cluster #this invokes kind create cluster
- make test # an apply against the kind cluster above
- make test-junit
services:
- docker
condition:
changesets:
includePaths:
*clsPaths

[lassian]

Hi Mariyan,

Yes the ability to specify your own dind image is coming in a future release I dont believe we have a public ticket for it as its being done as part of the GA release of runners.

Kind Regards,
Nathan Burrell

[mariyandimitrov]

Sounds awesome @lassian , thanks for the reply, look forward to this being available!

Just to clear things up in my head, in the snippet above, is my assumption correct that the docker service in the services section references https://hub.docker.com/r/atlassian/pipelines-docker-daemon to provide a DinD and the runner itself imposes security restrictions (or best practices, depends on point of view I guess) via plugins to deny having privileged containers, specific volume mounts (outside of BITBUCKET_CLONE_DIR boundaries) etc.?

Thanks again and looking forward to the GA!

Kind Regards,

Mariyan

[lassian]

Hi Mariyan,

Yes those assumptions are correct.

The initial release of runners (beta) we are just aiming for feature parity with our cloud runner before we start relaxing some of the restrictions we have to have in our cloud (due to its multi tenated nature) for self hosted runners (due to their single tenanted nature).

Kind Regards,
Nathan