Create
cancel
Showing results for 
Search instead for 
Did you mean: 
Sign up Log in
Celebration

Earn badges and make progress

You're on your way to the next level! Join the Kudos program to earn points and save your progress.

Deleted user Avatar
Deleted user

Level 1: Seed

25 / 150 points

Next: Root

Avatar

1 badge earned

Collect

Participate in fun challenges

Challenges come and go, but your rewards stay with you. Do more to earn more!

Challenges
Coins

Gift kudos to your peers

What goes around comes around! Share the love by gifting kudos to your peers.

Recognition
Ribbon

Rise up in the ranks

Keep earning points to reach the top of the leaderboard. It resets every quarter so you always have a chance!

Leaderboard

Come for the products,
stay for the community

The Atlassian Community can help you and your team get more value out of Atlassian products and practices.

Atlassian Community about banner
4,463,400
Community Members
 
Community Events
176
Community Groups

How to handle concurrent incidents with automatic incident rules defined

I have stumbled into an issue where my team (incident management) receives an alert for an incident from team A and while managing this incident team B also needs to start an incident so they send an alert to our team as well. Team B alert get's automatically associated to team A's incident.

We rely heavily on slack so this is where most alerts originate from. We tried to make the alerts unique by adding the message as a tag but that doesn't seem to work.

 

We are open to suggestions.

 

Thanks in advance!

1 answer

1 accepted

2 votes
Answer accepted

Hi Coltin, 

 

Justin here, from Opsgenie support - happy to help!

 

If I'm understanding correctly, it sounds like you have multiple groups channeling alerts to a single team within Opsgenie ('Incident Management'), and that Incident Management team is what contains the rules to spin up new incidents - is that correct?

 

If so - that may be problematic, as Opsgenie is set up to separate these incident rules out by team. Each team defines its own incident rules, and those rules are only evaluated for alerts assigned to that same team. 

 

So, one potential solution would be to set up team A and team B within Opsgenie, have them each define their own incident rules, and then route each team's alerts to their team in Opsgenie only, resulting in individual incidents for each team.

 

The other option would be to leave things as they are, and set up some additional filtering on the incident rule/s defined in the 'Incident Management' team. There's not currently a way to dynamically create new incidents, if, for example, there's a unique tag on the triggering alert, but you could add some data regarding the source of the alert (perhaps the team name), and then filter based on that value in the incident rule. 

 

There are potentially other options, but these are the two that jump out as the easiest to implement. However, if you have other questions about this, please feel free to reply here, or start up a new support ticket/chat with us - we'd be happy to help!

 

Best, 

 

Justin S.

Atlassian Cloud Support

@Coltin Rose How did you guys end up resolving this issue? I am running into a similar issue as well.

@Sterling Houghton We opted to move from all the incident rules being defined in our team to having them defined for each team in OpsGenie. We then updated our internal docs to include adding our team as a responder if in fact the incident that was created is a true incident. This helped with the transition while the teams were adjusting their alerting so false incidents weren’t reported and prevented alert fatigue with my team getting paged for non incidents.

hope this helps!

Suggest an answer

Log in or Sign up to answer
DEPLOYMENT TYPE
CLOUD
TAGS

Atlassian Community Events