Come for the products,
stay for the community

The Atlassian Community can help you and your team get more value out of Atlassian products and practices.

Atlassian Community about banner
4,298,120
Community Members
 
Community Events
165
Community Groups

Self-hosted runners - use a local dind

Hello,

We are setting up self-hosted runners to run the scripts on a hosted server instead of using the docker hub. We tried to do some tests to see what's best in terms of performance.

Ideally we would like to run the scripts on the machine where the runner is set up (this computer has all dependencies and libraries required to execute bitbucket pipelines). But I dont' think this is possible.

Furthermore, we would like to copy the compiled files from the pipeline to the server (to have local backups of the releases automatically).

Another alternative we tried is using the docker dind (docker in docker). We have docker images created for each project and uploaded on Docker Hub, and can use this feature to run a project docker image inside the docker runner container. For this option, it would be ideal if docker could load the docker image directly from the server (if downloaded previously already) instead of connecting with Docker Hub and downloading the image each time the pipeline is run (this would prevent more pipeline execution time).

Any idea of how to do that? Maybe adding some additional commands when setting up the docker-runner container? 

1 answer

0 votes

Hi Laura,

Ideally we would like to run the scripts on the machine where the runner is set up (this computer has all dependencies and libraries required to execute bitbucket pipelines). But I dont' think this is possible.

A runner doesn’t provide access directly to the host file system. However, it is possible to use pipes, so you could use e.g. the following pipe to run a script on your server:

Furthermore, we would like to copy the compiled files from the pipeline to the server (to have local backups of the releases automatically).

For this purpose, you could use rsync, scp, ftp, or another protocol in order to transfer the files from the runner to the server.

Another alternative we tried is using the docker dind (docker in docker). We have docker images created for each project and uploaded on Docker Hub, and can use this feature to run a project docker image inside the docker runner container. For this option, it would be ideal if docker could load the docker image directly from the server (if downloaded previously already) instead of connecting with Docker Hub and downloading the image each time the pipeline is run (this would prevent more pipeline execution time).

I'm afraid that this is not possible, the image needs to be hosted in a docker registry that is accessible on the internet.

If you have any other questions, please feel free to let me know.

Kind regards,
Theodora

Suggest an answer

Log in or Sign up to answer
DEPLOYMENT TYPE
CLOUD
TAGS
Community showcase
Published in Bitbucket

Git push size limits are coming to Bitbucket Cloud starting April 4th, 2022

Beginning on April 4th, we will be implementing push limits. This means that your push cannot be completed if it is over 3.5 GB. If you do attempt to complete a push that is over 3.5 GB, it will fail...

2,224 views 2 9
Read article

Community Events

Connect with like-minded Atlassian users at free events near you!

Find an event

Connect with like-minded Atlassian users at free events near you!

Unfortunately there are no Community Events near you at the moment.

Host an event

You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events

Events near you