Turn off Rate limit on REST API calls

[Jozef Burgel]

Is it possible to turn off rate limits ( https://confluence.atlassian.com/bitbucket/rate-limits-668173227.html ) for certain user off? We have integrated automatic sonar analysis on pull requests we create via bitbucket using REST API for one specific user . However the problem is that with more projects we are integrating we are experiencing problems with rate limits for the given account .

Would the creation of multiple user accounts used for this purpose solve this problem? Or is this "global" limitation no matter which users invoke the call? Is it somehow possible to turn this limitation off? 

[Sumanta Chakraborty]

This is a sad state of affairs that one cant even pay to get higher API limits. For the webhooks solution - its not a complete solution: To complete it, you should provide a client we can keep inside the network to talk to a service on the BitBucket Cloud over web sockets or long pole whatever to avoid punching a hole in the firewall. Something like https://docs.webhookrelay.com/ .

[Ana Retamal]

Hi Jozef! Those limits are set per user account and it's not possible to turn them off as their purpose is to keep the health of our platforms and prevent abuse. 

If you create a separate user account for some of the projects, each of the accounts will have separate rate limits so you'll be able to run more tasks simultaneously.

Hope this clarifies your question!

Ana 

[red888]

Is there any will to fix this? As it stands right now bitbucket cloud is not designed well for medium/large companies for CI\CD because of this limitation

I would gladly by more to increase the rate limit of the user my build server and sonar server uses- not turn off just increase!

Trying to create and balance multiple users to get around quota limits is not a very professional solution!

[Ryan Wischkaemper]

Agreed! Would gladly pay for more quota for a single user than have to manage multiple users

[Vivekanand G]

Any workaround to fix this?

[Michael Walker]

Hi Vivekanand and others,

I would love to hear the use cases where an increased number of API calls is necessary. The most common use for large amounts calls is typically seen when CI is polling Bitbucket for changes, however, we've since released webhooks that allow Bitbucket to notify CI of a change rather than the CI having to constantly, and often wastefully, poll Bitbucket for the changes. Read more about Webhooks vs Polling here.

With the above major hurdle of CI integration solved, what other needs do you have for bulk API calls?

- Michael

[Vivekanand Gavini]

we have a few services which modify git files directly. Also our CI system notifies the build status etc., to bitbucket.

We seem to be hitting this limit on a daily basis. Any workarounds would be really helpful. Would be glad to pay to increase quota than manage multiple users.

[brett_gardner]

Hi Michael

My use case is

• Multiple repositories
• Each repository has multiple deployments
• Each deployment has 3 or 6 variables (6 variables for our production deployments)
• These variables contain temporary AWS authentication variables that need to be updated every 12 hours

So my script has to retrieve deployments, retrieve variables and update variables and I am hitting the rate limit.

[Slavek Jurkowski]

+1 ..medium to large org here.. would gladly pay more to remove or increase limits! There should be an option that can be set so that service account users (non humans) using the bitbucket api can have higher rate limits.  It is ridiculous that we have to create multiple users as a work around ..how does that protect your api from high rate limits? if someone can just create multiple users to circumvent this?

[Ram Tandukar]

hi @Michael Walker

We are using https://jenkins-x.io to manage our CICD and GitOps flow. Jenkins X lighthouse keeper module polls, open pull request and other /repositories/ request path every second or so. Adding 10+ projects or so this would hit the 1000 per hour limit pretty quickly. Jenkins X also uses webhooks but in a different capacity. If you want to build closely integrated tooling around your git workflow, then there is a likelyhood people are are going to hit this limit pretty quickly especially in larger corporate organisations . Agree with the above comment, we will probably just add more users to circumvent this but is there must be a better/smarter way to manage this? Maybe the resource quote ? https://bitbucket.org/blog/living-the-value-dont-the-customer-in-bitbuckets-incident-response

Remember, “Don’t #@!% the customer” ;)

Ram 

[Crafton]

Hi @Michael Walker ,

We use Spotify Backstage (https://backstage.io/) internally and have a bit over 300 repos and over 120 users in our BB workspace, not sure if this is classed as medium or large. Backstage works by polling each repo for a catalog file when it's in discovery mode to determine if anything has changed. We use up the quota quite quickly when polling every 10 minutes. Multiple accounts won't work for us as Backstage cannot be configured in this way (AFAIK). This is currently a show stopper for us so if you could get back with potential options it would be greatly appreciated.

Github has a rate-limit of 5000 per hour (https://docs.github.com/en/rest/overview/resources-in-the-rest-api#rate-limiting) which I think is workable. Perhaps a good approach, which I think would also aid customer retention, may be to match them with limits while you build out the billing infrastructure necessary to manage per-customer api quotas. 

Regards,

Crafton