Segregate Bitbucket runners for each branch in Bitbucket pipeline with shareable steps

Satya Kesanakurty

Hi Team,

We are connecting to a database and making deployments to it from bitbucket pipelines and we have plenty of pipelines in place among which some are scheduled.

We have branch pipelines for many branches where one of the steps in each branch is common. So, we declared a shareable step and calling it in each branch pipeline.

We are making use of Bitbucket self-hosted runners. Earlier, we had a single self-hosted runner because of which only one pipeline used to run at a time. 

Later point of time, we added multiple self-hosted runners for making the pipelines fast. But we observed that, two pipelines of the same branch (one is scheduled and other is manual) started running at the same time because of available runners which is resulting in two processes accessing the same object in Database and thereby getting deadlock errors.

To prevent two pipelines in the same branch running at the same time, we thought of separating runners for each branch using Labels for runners and runners section in bitbucket pipeline. So that only one pipeline in a branch would be active at a time, and other pipeline in that branch would get into queuing state.

I tried to parameterize runners section in Bitbucket pipeline in the shareable step to see if the pipeline can pick runners at run time which isn't working.

Could you let me know if there is any chance to parameterize runners in shareable step and to decide the runner at pipeline runtime depending upon the branch? 

Please help me with this. Thanks in advance.

Thanks,

Satya.

Theodora Boudale

Hi Satya,

You could make use of the deployments feature in Pipelines instead of using a separate runner per branch. You can find info about this feature here:

• https://support.atlassian.com/bitbucket-cloud/docs/set-up-and-monitor-deployments/
• https://support.atlassian.com/bitbucket-cloud/docs/concurrency-control/

If a step with the keyword deployment, e.g., to production, is triggered while another step with a deployment to production is already running, then the second step will be automatically paused. Please keep in mind that paused deployments are not automatically resumed, they need to be manually resumed by a user at a later time.

 

If you prefer to use two different runners, this should also be possible. I am sharing below a sample yml file. A YAML anchor is used for a step, and then in the pipelines section, you can see how I am setting a different runner for each pipeline's step via labels. This assumes you have two separate runners, and branch1 and branch2 are labels you have added to each respective runner.

image: atlassian/default-image:4

definitions:
  steps:
    - step: &mystep
        script:
          - echo $BITBUCKET_BRANCH

pipelines:
  branches:
    branch1:
      - step:
          <<: *mystep
          runs-on:
            - self.hosted
            - linux
            - branch1 
    branch2:
      - step:
          <<: *mystep
          runs-on:
            - self.hosted
            - linux
            - branch2

Please feel free to reach out if you have any questions!

Kind regards,
Theodora