Best practice for multiple environments with same alerts

Question

I'm new to opsgenie and looking for the best way to structure my alerts and notifications.

My alerts are coming from azure alerts and we have different subscriptions sending alerts, we have different subscriptions for each env (e.g. test, uat, prod) - the alerts are the same definition in azure just a difference scope.

So ideally I'd like to only configure the alert integration once in opsgenie but I need to do different actions based on the subscription - e.g. prod should notify support + sre teams, but test should only notify the sre team and only in hours.

What is the best way to do this, I can see I can have a separate integration for each env but then if I want to change my alert setup (e.g. what goes in the title or description) I have to change it for each integration which seems wrong.

Any help, docs, examples would be appreciated!

Nick Haller · Answer

Unfortunately not when it comes to "set one set of fields which applies to all filters."

But, Create Alert actions can be duplicated - which essentially clones the action's filter, along with any adjustments made on the alert fields too. For example:

https://www.screencast.com/t/SlVpQsGs

I made several adjustments in default Create Alert action. When I duplicated it, all the changes made also were added into the additional action.

Matt Browne_54gt44 · Answer

many thanks that is helpful and is a route I have gone down, but the disadvantage I see is that anything I set in the "Alert fields" is specific to the filter, e.g. I can't set one set of fields which applies to all filters - is that right?

Ideally I'd like one set of fields as the only thing that differs on the alert is the filter, I want the alert to look the same regardless of the filter if that makes sense.

I am on the standard plan btw.

Nick Haller · Answer

Hi @Matt Browne ,

I believe this can done under one integration - as long as you are subscribed to a Standard or Enterprise plan. These plans offer the Advanced tab within an integration that allows for this type of filtering: https://docs.opsgenie.com/docs/advanced-integration-settings

You'll first want to assign this integration to [No Team] - which allows for more than one team to be added as a responder. Then navigate into the Advanced tab"

The Advanced tab provides the flexibility and functionality to filter on what data / requests will create alerts, as well what teams / responders will be notified during creation.

Depending on what field parses the Azure subscription is what you will filter on for each action, as well what teams will receive these alerts. Here are a few examples if in Subscription ID were to be used:

When it comes to only notifying users during business hours - this cannot be done within an integration, but can be configured under the team's On-Call tab.

A Routing Rule could be configured under the SRE team to filter on what field parses the Azure subscription, as well be restricted to time intervals - then route these alerts to an escalation path, or No-One if created during after hours. That might look something like this:

Hope this helps. Let us know if you have additional questions.