Multiple Runners per Repository — Only One Runner Executes

[Leonardo Dadic]

In my case, I have a single repository that needs to be deployed (via git pull) to multiple different servers/mandanten. Currently, I have set up three self-hosted Linux shell runners — one per server — but every time I trigger a pipeline, only one runner executes the job, usually at random. As a result, only one of the servers gets updated.

In my bitbucket-pipelines.yml file, I have just one step, and all runners are registered with the same labels.

My questions are:

1. Is it possible to run the same pipeline step on multiple self-hosted runners for a single repository?

2. If yes, what is the correct or recommended configuration for this setup?

3. If not, what would be the best workaround or best practice to deploy the same code to multiple target serversfrom a single pipeline?

Thank you in advance for your help!

[Theodora Boudale · Accepted solution]

Hi Leonardo and welcome to the community!

This is expected behavior. If you have one pipeline definition, then one build will run, and it will run on one of the available runners.

There are a few ways to achieve what you want. I am using a numbered list below to share the options (I believe option 3 may be the simplest one).

1) You could use one runner for e.g. server1, and then from server1 connect to each of the other servers, e.g. via SSH, and perform a git pull on the repo directory on these servers. You would need to configure SSH access among the servers (not sure if that's desirable) and adjust your bitbucket-pipelines.yml file to add commands in the script for the ssh connection and the command you want to execute.

2) If that's too troublesome to configure or if you don't want to configure SSH access among the servers, then you can do the following:

• Edit your 3 runners from Repository settings or Workspace settings (depending on where you created them) and add to each one a unique label, e.g. server1, server2, server3.
  
  
• In the bitbucket-pipelines.yml file of your existing pipeline:
  
  
  • Adjust the the runs-on parameter and add the label server1, so that this step will always run on server1.
    
    
  • Add two custom pipelines, one of them to run on server2 and one of them to run on server3 based on the labels in the runs-on parameter. Add a script that will perform the deployment on each of these servers. Custom pipelines do not get triggered automatically.
    
    
  • In the script of your existing pipeline, you could use our API (e.g. with curl, if curl is available on server1) in order to trigger the two custom pipelines. There is an example on how to trigger a custom pipeline here (you can omit the variables, if your custom pipelines don't have variables).

This way, your existing pipeline will always run on server1 and deploy there, and then trigger the two custom pipelines, so there will be two additional builds which will run on server2 and server3 (because of the labels).

3) You could also simply add two more steps to your existing pipeline.

You can give a unique label to each of the 3 runners (from Repository settings or Workspace settings, depending on where you created them), and then add the respective labels to the runs-on parameter of each step in the bitbucket-pipelines.yml file.

So, the first step would always run on server1, the second step would always run on server2, and the third step would always run on server3.

Would any of this work for you? Please feel free to let me know if you have any questions.

Kind regards,
Theodora

[Leonardo Dadic]

Hey Theodora,

thanks a lot for your response. The solution with the additional steps works best for me!

[Theodora Boudale]

Hi Leonardo,

That's good to hear and you are very welcome!

I just wanted to add, you can make the three steps parallel:

• https://support.atlassian.com/bitbucket-cloud/docs/parallel-step-options/

This way, you won't need for the deployment on one server to finish before the other one starts. Additionally, with parallel steps, if the deployment on one server fails, the other two parallel steps will continue.

Kind regards,
Theodora