Self-hosted runners - use a local dind

[Laura Pons]

Hello,

We are setting up self-hosted runners to run the scripts on a hosted server instead of using the docker hub. We tried to do some tests to see what's best in terms of performance.

Ideally we would like to run the scripts on the machine where the runner is set up (this computer has all dependencies and libraries required to execute bitbucket pipelines). But I dont' think this is possible.

Furthermore, we would like to copy the compiled files from the pipeline to the server (to have local backups of the releases automatically).

Another alternative we tried is using the docker dind (docker in docker). We have docker images created for each project and uploaded on Docker Hub, and can use this feature to run a project docker image inside the docker runner container. For this option, it would be ideal if docker could load the docker image directly from the server (if downloaded previously already) instead of connecting with Docker Hub and downloading the image each time the pipeline is run (this would prevent more pipeline execution time).

Any idea of how to do that? Maybe adding some additional commands when setting up the docker-runner container? 

[Theodora Boudale]

Hi Laura,

Ideally we would like to run the scripts on the machine where the runner is set up (this computer has all dependencies and libraries required to execute bitbucket pipelines). But I dont' think this is possible.

A runner doesn’t provide access directly to the host file system. However, it is possible to use pipes, so you could use e.g. the following pipe to run a script on your server:

• https://bitbucket.org/product/features/pipelines/integrations?p=atlassian/ssh-run

Furthermore, we would like to copy the compiled files from the pipeline to the server (to have local backups of the releases automatically).

For this purpose, you could use rsync, scp, ftp, or another protocol in order to transfer the files from the runner to the server.

Another alternative we tried is using the docker dind (docker in docker). We have docker images created for each project and uploaded on Docker Hub, and can use this feature to run a project docker image inside the docker runner container. For this option, it would be ideal if docker could load the docker image directly from the server (if downloaded previously already) instead of connecting with Docker Hub and downloading the image each time the pipeline is run (this would prevent more pipeline execution time).

I'm afraid that this is not possible, the image needs to be hosted in a docker registry that is accessible on the internet.

If you have any other questions, please feel free to let me know.

Kind regards,
Theodora