Bitbucket Rate Limit during clone, fetch

[Andrew Liew]

Hi,

We are currently facing a build fail by Jenkins when it attempts to fetch the repositories. For a few projects that has data size less than 2Mb, the operations went well. When Jenkins attempt to fetch projects that has data size of 100Mb or less, the operation fails.

Is there a rate limit for project with large data size ?

Thanks,

Andrew

[Patrik S]

Hello @Andrew Liew ,

Thank you for reaching out to Atlassian Community.

We do have some API rate limits implemented to grant stability across our platform, but those limits are based on the number of requests, and not on the size of the repository. The number of API calls is measured as follows: 

• Unauthenticated calls: measured against a specific IP address.

• Authenticated calls: measured against the user ID.

The limits of each kind of request are outlined in the following documentation : 

• https://support.atlassian.com/bitbucket-cloud/docs/api-request-limits/

Having said that, if your Jenkins instance is using the pooling method to check for changes in the repo, meaning that it will be frequently calling the bitbucket API to check if a new build should be triggered, the user used by Jenkins to authenticate might be affected by the API rate limiting, depending on the number of calls per rolling hour. When a user is affected by rate limiting, we return a 429 HTTP status code to the request.

If that is the case, to avoid this limitation you can either reduce the pooling rate of your Jenkins instance, so it calls bitbucket API less frequently, or implement the usage of Bitbucket Webhooks, so bitbucket itself will send a request to your Jenkins instance when a given event happened to the repository (for example a new push)

Checking our internal tools I've noticed you have access to a workspace in a Premium subscription, so in case you want us to check our internal logs to confirm if your Jenkins requests are being rate limited, you can raise a support ticket with us in the following link : https://support.atlassian.com/contact/#/ 

Thank you, @Andrew Liew !

Kind regards,

Patrik S