Rate Limiting for the API - why do I see regular user accounts?

[__ Jimi Wikman]

We have recently activated Rate Limiting, and I am a bit surprised to see so many regular user accounts that show up, and I wonder why that would be?

I also wonder what could hide in the anonymous account, as that is a bit vague for me in the description of rate limiting.

[Sunny Ape · Accepted solution]

"I wonder why that would be"

... because you enable Rate Limiting. That was the whole purpose of enabling it right? To find out who has been using the REST API without limits.

"I wonder what would could hide in the anonymous account"

Nothing 'hides' in it, it's just an abstract account, as it says in the documentation...

Anonymous: This entry gathers all requests that weren’t made from an authenticated account. Since one user can easily use the limit for anonymous access, it might be a good idea to add an exemption for anonymous traffic and give it a higher limit.

 If it really bothers you, disable anonymous access and be done with it. Speak to your Jira sys admins and they'll explain it to you better.

[__ Jimi Wikman]

No need to be rude when people ask questions. It is ok if you don't know the answer, just don't reply if you have nothing helpful to add.

Of course, I will see what users are consuming data through the API, but none of the users that show up have anything to do with the API. So do they show up because of automations, apps...it does not make any sense to me.

What about Slack or Teams? Do they consume on an individual level?

Anonymous: This entry gathers all requests that weren’t made from an authenticated account. Since one user can easily use the limit for anonymous access, it might be a good idea to add an exemption for anonymous traffic and give it a higher limit.

This tells me nothing. What could connect to the API and consume data without an account? That makes no sense to me, so I would block it unless I know what it is. So, does anyone know what it could be anonymous so I know what is being blocked?

[Sunny Ape]

I'm sorry that you interpreted my answers as being rude.

Perhaps you'd be better off liaising with your company's Jira systems admins. They have the training and technical understanding to know how the product works, know what data flows through the REST APIs, which products and components interact with it and why those interactions would show up for 'regular users'.

Those sort of people would never turn on Rate Limiting without reading the documentation first to know exactly how it works and how to use it properly.

Good luck.

[__ Jimi Wikman]

I am the system admin and I have read the documentation.

The problem is that the documentation does not mention why personal accounts would show up as consumers of the API and, as you already copied in, the anonymous description does not say anything what an anonymous request could be.

Hence, my question.

Now if you do not have the answer to this, then let someone that do answer please? Preferably one of those system admins that know everything that you refer to :)

[Radek Dostál]

I'd say David's reply covered it precisely.

You can see the requests from accounts in tomcat's access log which would tell you which endpoints those users are calling. From the endpoints you can make some guesses as to what they are trying to do, you can also see the user agent or other headers in load balancer's logs depending on your config to make some more guesses about what's going on.

If you're using web interface, you should not be under rate limiting at all, since it only applies to remote calls outside of browser. Thus, those users did fire those requests remotely - otherwise this would be a bug in rate limiting.

Then again, everything here is explained in the documentation https://confluence.atlassian.com/adminjiraserver/improving-instance-stability-with-rate-limiting-983794911.html, especially what anonymous requests mean.

[__ Jimi Wikman]

Thank you for your reply.

Would you mind showing me where on that page you see any explanation to what anonymous usage is? Because I only see two sections about it:

Recommended: Add an exemption for anonymous access

Jira sees all anonymous traffic as made by one user: Anonymous. If your rate limits are not too high, it might happen that a single user drains the limit assigned to anonymous. It’s a good idea to add an exemption for this account with a higher limit, and then observe whether you need to increase it

This is an advice that require you to understand the impact of setting the exemption, which require you to know what could cause the anonymous requests to begin with?

Anonymous: This entry gathers all requests that weren’t made from an authenticated account. Since one user can easily use the limit for anonymous access, it might be a good idea to add an exemption for anonymous traffic and give it a higher limit.

Self explaining that it comes from unauthorized usage, but I still have no explanation as to what could be the source or how they can consume data without authorization?

So that does not answer my question, but if you have a section where it is explained, please link it.

You can see the requests from accounts in tomcat's access log which would tell you which endpoints those users are calling. From the endpoints you can make some guesses as to what they are trying to do, you can also see the user agent or other headers in load balancer's logs depending on your config to make some more guesses about what's going on.

This just tells me you don't know either why regular user accounts show up and you are asking me to look into the server logs and guess? I did not ask for suggestions on where to go to look for guesses, I was asking for clarifications so I did not have to guess.

Listen, I understand you are trying to help the best you can, but I am not looking for guesses, I want to understand exactly what rate limiting is doing and why it shows things I did not expect to see from API interactions, such as regular user accounts that should have nothing to do with the API.

I am expecting that there is something internally that uses the API, such as Slack/Teams integrations or an app, but I need to get this confirmed since the rate limiting does not provide any information on what is happening.

Surely someone must know how rate limiting works and have seen the same behavior before?

[Radek Dostál]

I don't know what you expect to hear, and I doubt you really know it either. If you need a third person to tell you the same thing, I'm glad to be getting notifications when that happens.

Alas, I will try this once more, even though I should know better but I'm stuck on a call so here goes:

know what could cause the anonymous requests to begin with?

An anonymous request means that the request didn't come with an authorization header, or came without any session cookie. What else could it be?

I still have no explanation as to what could be the source or how they can consume data without authorization?

Here's a question for you, if you make a request towards an endpoint without any authentication, does Jira receive the request to handle, or does it get auto-blocked by reverse proxy?

Because Jira has both anonymous as well as authorization-required endpoints, it must be able to accept both anonymous as well as authorized calls. Thus, and I don't know what other words to use here, anonymous requests come from anonymous requests. There's no magic or hidden meaning here, they're anonymous requests.

If you still need a practical example:

 - a client running an http request without any authorization headers

 - web crawlers, such as google, or by companies for e.g. intranet search results

 - vulnerability scanners, wild on the internet, or by companies scanning their own network

you are asking me to look into the server logs and guess?

If you expect Jira to analyze the nature of user behaviour and intentions, then you're using the wrong tool for the job. Obviously, we cannot tell you what your users are doing, and neither can you - and you won't know that without logs either.

If you don't want to consult the logs, go ask the users. If you refuse to look at the IP addresses, user agents, timestamps, endpoints, then there is nothing anyone can do and this will remain a mystery. If you say those users should have nothing to do with APIs, I won't be trying to convince you otherwise.

I am expecting that there is something internally that uses the API, such as Slack/Teams integrations or an app, but I need to get this confirmed since the rate limiting does not provide any information on what is happening.

This is a contradiction. Teams or Slack do not live inside Jira, thus they are external apps. If you want to integrate with them, then they are going to be doing remote requests. How those requests are authenticated depends wholly on how the integration is configured and what it allows.

[__ Jimi Wikman]

Please Radek, I understand you mean well, but you either don't understand what I am asking or you are just trolling me right now. I will assume it is me that is not explaining things properly.

An anonymous request means that the request didn't come with an authorization header, or came without any session cookie. What else could it be?

Of course. I know what an unauthorized request is, but since we are on prem with the API restricted in our network, this is not from an external source. It is an internal source, but we don't have anything that is not an app or through a named service account. Since my name is one of the names that comes up and I am very sure I do not make any HTTP requests towards the endpoint, I am asking what could cause this behavior as the only explanation I can think is that it is something inside Jira that is causing this.

This is a contradiction. Teams or Slack do not live inside Jira, thus they are external apps. If you want to integrate with them, then they are going to be doing remote requests. How those requests are authenticated depends wholly on how the integration is configured and what it allows.

Slack integrations are built into Jira. It has been ever since Slack bought HipChat. I assume that this integration is built with a proper Oath connection with a named service account?

If you still need a practical example:

- a client running an http request without any authorization headers

- web crawlers, such as google, or by companies for e.g. intranet search results

- vulnerability scanners, wild on the internet, or by companies scanning their own network

Now this is useful. Scanning the network we have a lot of, but they should never have to scan our API.  This I can work with and examine further.

Thank you, this is the kind of help I am looking for.

If you expect Jira to analyze the nature of user behaviour and intentions, then you're using the wrong tool for the job. Obviously, we cannot tell you what your users are doing, and neither can you - and you won't know that without logs either.

You misread this. The key wording I used was "guess". This as you suggested me to do just that, look at the log and guess what is happening. I can do that, but I would rather have answers than guessing from logs that is also not very useful as you will see things like:

Jira.exceptions.JIRAError: JiraError HTTP 429 url: URL HERE
text: Rate limit exceeded.

What I was hoping for is someone that knows how rate limiting actually work since there is no documentation on it other from a user perspective. I guess no one in the community actually knows, so I will turn to Atlassian instead for answers.

[__ Jimi Wikman · Accepted solution]

If anyone else run into this, the reason why you see individual users and why you have a high number of anonymous usage is because apps and internal functionality is not compatible with the rate limiter at this time.

If you want to use it to just prevent users hammering the api to oblivion, then it works fine. You can't turn on the feature to block any api connection that is not register as an exemption, however. That will just break your solution.

I hope that helps someone else wondering about this, as it is not described or mentioned anywhere in the documentation that I have found.