nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Is there a way to programatically create a new runner? (Not using the Bitbucket web interface)

kalib

Hi, is it possible to create a runner in any programmatic or automated way? I know we need to go to the web interface and go to the settings and create a runner, copy that docker command and start that command in a host. I was wondering if there's a way to do that automatically, maybe via an API call or something. The idea is that I wouldn't need to have dozens of runners up and running 24/7 and to be able to spin up runners at specific times or even as a step of some pipeline for example.

But since bitbucket pipelines requires that specific ID for each runner that is generated only during the creation of the runner, I was wondering if there would be a way to do that via command line somehow, so I can automate the creation of the host and starting the docker container there automatically.

Accepted answers

All comments

Mark C

Hi @kalib

Thank you for reaching out to the community.

Unfortunately, it's not possible to create Pipelines Runners programmatically using REST API at the moment.
We do have an existing feature request for it that can be located through this link. - https://jira.atlassian.com/browse/BCLOUD-21708
You can upvote and watch it for now so that you'll be notified of any updates from our team when the feature becomes available on Bitbucket Cloud.
Please do note that we don't have an exact ETA for the feature request as all new features will be implemented according to our policy here.

Let me know if you have further questions that I can help with.

Regards,
Mark C

Einar Coutin

Thanks for your policy document. So if the permissions for managing runners are already implemented in the API:
https://developer.atlassian.com/cloud/bitbucket/rest/intro#runner

And there are TWO tickets requesting it:
[BCLOUD-21309] Have public API endpoints for pipelines runners - Create and track feature requests for Atlassian products.

[BCLOUD-21708] API for creating a new Runner - Create and track feature requests for Atlassian products.

Were there already plans to do this before it was requested?

The use cases are pretty obvious from an auto-scaling perspective, I mean since it has already been mentioned in these forums that there's a requirement of setting docker in docker for the bitbucket runners combined with the fact that they're configured via an UUID at creation time rather than at registration time makes it all more cumbersome to be launching docker containers/runners with a FIXED UUID environment variable that comes from the api when the runner is registered rather than registering the runner with a UUID given by your cloud endpoint after a slot for it has been created in the workspace/repository when it calls it.

With that said, we understand that Bitbucket Cloud Runners(hosted by you) Autoscale automatically. But two limitations are present:
1.-There's no On Prem Connectivity outside of a DMZ.
2.-The limitation of only having up to 8GB of RAM limits runner usage.

Combine those with the fact of how the Self-Hosted runners flow works (where the registration token is created and given to the user AFTER the fact) and there's not much we can do to auto-scale self-hosted runners, even if we created an API outside of Bitbuckets: for example AWS Lambda and Cloudwatch, it wouldn't work because runners can't use the aws-logger docker driver to send data to Cloudwatch, which makes sense since the json-logger driver is required in order to collect and send data to Bitbucket cloud.

Mark C

Hi @Einar Coutin

I totally understand that.

Indeed, those BCLOUD tickets you mentioned are identical.
Hence, I went ahead and marked the latter as a duplicate.

Regarding Bitbucket Cloud Pipelines Runners auto-scaling, we have an existing feature request for it here where you can vote/watch as well: https://jira.atlassian.com/browse/BCLOUD-21343

Alternatively, if you're using Kubernetes, we've recently shipped a Kubernetes Autoscaler for Runners that is currently in Early Access Preview (EAP).
You can join the group here for more details on how to use it: https://community.atlassian.com/t5/Bitbucket-Pipelines-Runner/gh-p/bitbucket-runner-autoscaler-4k8s

Regards,
Mark C

Einar Coutin

Thanks for your reply Mark!

I've requested to join the group. Regardless of autoscaling environment (whether Kubernetes, EC2, etc) the question remains as to how are the runners registering against the Bitbucket cloud database in order for pipelines to identify them and use them.

You can't put the cart in front of the ox so to speak. And right now there's no way to get a runner's UUID without registering MANUALLY on the API first. That's the challenge, not creating copies of the agent.

Do you have any insights into that part of the solution?

Thank You!

jawad_abbasi

Hi @Mark C what's an update about that ticket to build some api to create bitbucket runner automatically ?

Welcome!

It looks like you're new here. Sign in or register to get started.

Quick Links

Find more posts tagged with

cloud

bitbucket

bitbucket-cloud

bitbucket-pipelines