418 Flagged as Scanner - What can I do?

[mraaroncruz]

I've built a Statuspage Status API poller (specifically to poll Influx Cloud Status) but I have been intermittently running into this error:

 {:bad_response, 418,
  "Your IP address [OUR IP ADDRESS] has been flagged as a scanner. Scanners are not permitted. If you are seeing this message in error, please contact us at https://support.atlassian.com/contact.\n"}}

I've looked for docs on rate limiting but they don't exist. All I've found is one page telling me that Status API is not rate limited: 
https://support.atlassian.com/statuspage/docs/what-are-the-different-apis-under-statuspage/

and a support ticket saying it is rate limited but not being specific about what that limit is:
https://community.atlassian.com/t5/Statuspage-questions/Github-runner-flagged-as-a-scanner/qaq-p/2224066

There is also no rate limit info in the response headers (here is the endpoint I'm hitting btw https://status.influxdata.com/api/v2/components.json).

So I have two questions:
1. What is the rate limit and what is a reasonable rate at which I can poll this status endpoint?
2. I see an Etag in the response, I have tried it out with an  `If-None-Match` header and it has properly responded with a 304. Are these requests subject to whatever is flagging me  as a scanner?

Note: I know that I can subscribe to updates but I would have to trust that I also got the "everything is ok" webhook and that my system wasn't restarting or unable to handle these which adds a lot of complexity to my service, and if don't receive these messages, then I'm in the same boat of needing to make a request to a service that considers me a "scanner"

Thank You!

[Jesse Klein]

Hello there,

Welcome, and thanks for reaching out to the community with your question about scanner IP banning. I don't have specific details on rate limiting. The Status API does not have a rate limit as far as I am aware. I can reach out to the other engineer to ask where he got that information. Checking how the scanner detection works, though, we actually use a lot of different methods to figure out if a scanner is being used. The only mention I see related to rate limiting is if you are trying to send requests to invalid domains, not existing ones. I also don't see anything related to etags.

For us to provide more information, I suggest opening a support ticket at support.atlassian.com so we can see what IP address was banned and give more information about that. The tool we use will show us the reason if we have the IP address. This would help us determine if we are collecting false positives and give us a reason to escalate this to our engineering team.

Thanks for your time!

Regards,
Jesse

[mraaroncruz]

Thanks Jesse,
We're using the API as described on the Influx API docs page (https://status.influxdata.com/api) and only polling a twice a minute.

This 418 status only comes sometimes as it works as expected most of the time which is strange if we're marked as a scanner, though of course I have no idea how the flagging system works.

I may still open a support ticket. 
Thanks for your time,
Aaron

[Jesse Klein]

Hi Aaron,

Thanks for that context. That seems like a normal use-case in terms of usage. There are many pieces to the flagging system so it would be hard to narrow it down. Here is what I recommend. If you get flagged again, the banner puts a temporary IP ban for 24 hours. It also logs the reason for a tool we internally have. If you do get banned again, definitely consider raising a ticket with us and we can see what the logs say for the reason of the ban. Thanks again for your time!

Regards,
Jesse