Hope everyone is well out there. This article will focus on the impact of Source Code Management (SCM) polling in Jenkins and Bitbucket Cloud Rate Limits.
What is SCM polling in Jenkins?
The plugin is designed to poll Bitbucket cloud repositories for changes based on the interval as specified by a user. The changes can include: new commits, branches, pull requests. This is a useful feature that many Bitbucket Cloud Customers use. Especially the Bitbucket Branch Source Plugin (installed approximately 65555 times)
What is API Rate Limiting?
In general, rate limiting is used to protect overuse of API, by limiting the amount of calls an authenticated user can make the the application over API. It also helps the provider prevent flooding and request queues that will cause delays caused by “noisy neighbours“ and to identify and deflect DDoS attack. Another application of rate limit is to make the API scalable, a sudden increase in the usage of a new/popular API can cause a traffic spike causing severe lags.
API Rate Limiting in Bitbucket Cloud
For current Bitbucket Rate Limits, please check our support documentation;
Now that we have some of the key information out of the way, let's get to the situation at hand.
Over a period of time, we have noticed our users who integrate their accounts with Jenkins set up SCM polling from the Jenkins side. When the amount of API calls reaches the limit due to polling per endpoint per user, Bitbucket will not accept any more calls and the response will be 429 to indicate that the user reached its limit per current hour. This causes frustrating issues such as pages not loading on time, having to refresh multiple times to see even the tiniest of data.
Since there is no explicit information on the UI mentioning API rate limit in place, this can lead to a frustrating day indeed. You might be greeted with multiple “something went wrong” error akin to this;
What we can do here is to open up our Browser’s network tab, developer console and look for a “429” error which would indicate Rate Limiting.
So this can help you identify what is happening, but that is not quiet enough, let’s see how we can work around while using SCM polling that is interacting with Bitbucket.
Adjust the Polling
If at all possible, the focus can be on increasing the time between requests. Adjust the polling in a way that it does not overwhelm the system with requests and triggers rate limiting from Bitbucket platform. If this is not possible, and if you need to do the polling at that rate, then we can look at the second option;
Set Up a dedicated CI/CD account in Bitbucket Cloud
Here, you would have to sign up for a new Bitbucket Cloud account, you can perform your Jenkins Integration and SCM polling to this account, this will make sure that your main individual account that you use for UI navigation and anything else in Bitbucket will not be rate limited.
One thing to remember here is that this new user will be counted towards your workspace plan. As I write this, this extra user a topic of discussion internally and any updates on this will be shared with all our customers.
Note: I hope this article provided you with an insight on why rate limiting happens on Jenkins SCM polling and the ways to work with/around it.
Finally, if you have any other issues, API or otherwise, you can always find us at Bitbucket Cloud Support
Stay Safe & Have a wonderful day!
Manu AbrahamAtlassian Team
Announced in this blog, this holiday season we’re celebrating all things CI/CD and between now and the end of 2019 we’ll be showcasing content, use cases, feature announcements and more. One featur...
Connect with like-minded Atlassian users at free events near you!Find an event
Connect with like-minded Atlassian users at free events near you!
Unfortunately there are no Community Events near you at the moment.Host an event
You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events