How to do maintenance on local runners

[Hans de Groot]

We are running local runners for Bitbucket pipelines. We have multiple, and I need to make them low effort to operate them. That means that auto updates have been enabled (including reboot) and I also want to auto update the agent itself. As the runners are Linux based and are run using docker, I plan to setup watchtower.

How do I make sure the update does not interrupt a (potential) running build?

[Theodora Boudale]

Hi Hans,

The first step before updating a runner is to stop the runner and doing so will also interrupt any step running on this runner.

You could use our API to get all repos in a workspace and all pipelines in a certain repo that have a status of PENDING, BUILDING, or IN_PROGRESS:

• https://developer.atlassian.com/cloud/bitbucket/rest/api-group-repositories/#api-repositories-workspace-get
• https://developer.atlassian.com/cloud/bitbucket/rest/api-group-pipelines/#api-repositories-workspace-repo-slug-pipelines-get

One of my colleagues posted in another community question an example API call with curl that returns in progress pipelines for a certain repo:

• https://community.atlassian.com/t5/Bitbucket-questions/Re-Re-Possible-value-for-status-filtering-in-API-pipelines/qaq-p/2071030/comment-id/83461#M83461

If this call returns running pipelines, you can delay the update and retry later when there are no running pipelines.

Please keep in mind that the results do not indicate if a certain step runs on a self-hosted runner or on Atlassian's infrastructure. If you want this info and you also want to know which runner is used, you could get the steps of the running pipeline, then get the log file of the running step and parse it to see if that step runs on a runner and if so, what is the runner's UUID. These are the API endpoints for retrieving the steps of a pipeline and the log of a certain step:

• https://developer.atlassian.com/cloud/bitbucket/rest/api-group-pipelines/#api-repositories-workspace-repo-slug-pipelines-pipeline-uuid-steps-get
• https://developer.atlassian.com/cloud/bitbucket/rest/api-group-pipelines/#api-repositories-workspace-repo-slug-pipelines-pipeline-uuid-steps-step-uuid-log-get

Alternatively, if you know that all your self-hosted pipelines are finished by a certain time in the day and they only start in the morning of the next working day, you could schedule the update during that timeframe of inactivity or during the weekend (if you have no pipelines running in the weekend). If you use scheduled pipelines, you would also need to take into account the time that you schedule them and the time they usually need to finish, so that you schedule the update accordingly.

Kind regards,
Theodora