Why is an alert created for the incident creator?

[Rodolfo]

If I create an incident from the UI without impacted services and no responder teams, just with a responder user, Opsgenie creates 2 alerts:

1. For the responder user (as expected).

2. For the user who created the incident.

Why does the incident creator has to acknowledge an alert? Am I missing something?

[Samir · Accepted solution]

Hi @Rodolfo ,

Yes - this is expected behavior. When you create an incident, it creates a Responder Alert for each user/team that is a responder on the incident.

When you create an incident manually, it also creates an alert for the person who created the incident. There isn't a way to change this behavior, but we have a feature request to be able to disable this. I can add you to that request.

Please let us know if you have anymore questions!

Thanks,

Samir

[Rodolfo]

Hi @Samir, thanks for your response.

Yes, please. Add me to the request.

Regards

[Lina Massarwa]

Hi @Samir,

What is the logic in front of this behavior? Why do we need these alerts' creation?

[Samir]

Hi @Lina Massarwa 

The reason is because if an incident is created without any services impacted, nor other teams/users added as responders, then the incident wouldn't notify anyone.

So this alert is created to ensure that at least 1 responder alert is generated for every incident.

Definitely understand how you would not want this alert created, since the user creating the incident wouldn't always need to be notified of the incident right away.

[Rodolfo]

This would easily configurable with (pseudo code):

if not service and not responder_team and not responder_user:
    create_alert_for_incident_creator.

And you could even add a feature toggle in settings.

[Samir]

The other benefit is it allows the user who created the incident to get updates on the incident, because when an incident is resolved, the responder alerts are acknowledged, and when the incident is closed, responder alerts are closed

We understand this behavior is not ideal for all users, so we have a feature request for the option to change/disable this alert that gets generated when incidents are manually created.

[Samir]

I have gone ahead and created a public suggestion for this as well.

You can watch and vote for this suggestion here: https://jira.atlassian.com/browse/OPSGENIE-16

Thanks,

Samir

[Lina Massarwa]

Samir, Rodolfo,

Thanks a lot for the information.

@Rodolfo where can I put this pseudo code?

[Lina Massarwa]

@Samir ''

I'm very impressed with the tool, but I still feel difficulties with some understanding and the logic of how the tool needs to work.

I can divide my concerns/questions into both parts:

1) Notification Management:

How do different users' roles (owner, responder, the user who ack) manage notification? Are there other situations that the users receive notifications? (unless they are on call schedule).

When do they stop receiving notifications?

Are there generated alerts about incidents?  

When do they stop receiving notifications?

2) Alert & incident management:

The aim is to see all the alerts/incidents related to the same service. For the incidents, no problem- we can see by the affected service?

But what is the best practice of relating all the alerts to the related service?

Sorry Samir, for the long questions... I try to understand the best practice and logic of Opsgenie.. I've read much documentation and seen videos but still no progress. I appreciate your help.

I 

[Samir]

Hi @Lina Massarwa ,

I'd recommend checking out our documentation and other resources on how Opsgenie notifies users.

There are a lot of components in Opsgenie that dictate who is notified of alerts.

But essentially, this is the flow...

1. Alerts are created through integrations - integrations dictate which team(s) the alert goes to 
2. When an alert reaches a team, it follows the routing rules which generally route alerts to an escalation policy
3. The escalation policy defines who is notified of alerts. Usually the first step is to immediately notify on-call user, then after X minutes, it will escalate and notify someone else

Each user determines how they are notified (email, sms, voice, mobile) via their notification rules they configure on their profile. So each user can specify their own notification rules.

As far as generated alerts from incidents - whenever an incident is created, Responder Alert(s) are created for each team that is a responder on the incident. Each responder alert goes to it's respective team and follows the routing rules on that team as detailed above.

Alerts are not tied to services in Opsgenie, only incidents are. Alerts are created from integrations and go to teams.

Incidents are more critical issues that could result in downtime or a service disruption - this is why you can select impacted service(s) for an incident, to represent which service(s) are affected. Incidents generally require multiple teams to collaborate to pin down the root cause, and resolve the issue, so they are larger issues rather than alerts which are your normal alerts from your monitoring tools, ticketing tools, etc.

[Lina Massarwa]

Hi @Samir 

great answer and details! thanks a lot!

Are there automatic and direct incident' creations from external integrations?

[Samir]

@Lina Massarwa You're welcome!

Integrations can only create alerts (with the exception of our Zendesk integration which is the only one that can directly create Incidents).

However you can set-up incident rules within Opsgenie to automate creation of incidents from alerts. So integration creates alert in Opsgenie > matches incident rule in Opsgenie > incident is created.

[eric.law]

We actually RELY on this feature to trigger external orchestrations upon incident creation, because (from what I can tell) integrations are only triggered by alerts.

I have not seen an integration that triggers at-least-once per incident.

As this is configurable, being able to disable it would be great (we've had several questions about it already), but altering this _feature_ without a config setting would break our orchestration enough to cause notable pain.