Create
cancel
Showing results for 
Search instead for 
Did you mean: 
Sign up Log in

How to temporaly silent opsgenie notifications?

Daniel Leite
Contributor
July 2, 2019

With the recent cloudflare network problem, our sites went down. Of course, opsgenie received alerts form several systems and kept beeping and showing up notifications  for all of then.

So the question is how to silent all notifications for X minutes, when we know already about the problem or a service is flapping.

They should still show in the site/app, but just do not show notification nor alerts... and of course, after the X minuted, any new alert should send notification again

 

Thanks for the help

3 answers

1 accepted

1 vote
Answer accepted
Nick Haller
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
July 2, 2019

Hi Daniel,

Thanks for reaching out. I think your best bet for this scenario would be a Maintenance Policy which allows you to enable or disable multiple policies/integrations with a single entry during a specified window.

If you would like the alerts created in your Opsgenie account, but notifications delayed/suppressed - you'll want this Maintenance window to enable a Notification Policy. If you would like the alerts to NOT be created you will want to disable an integration - obviously if an alert isn't created, no user(s) will be notified.

Both the Notification and Maintenance policies can be set on the team level under the Policies tab. If you were to lean towards enabling/disabling a Notification policy, you will first want to create this.

Within the filters of the Notification policy - you can select ALL alerts to be delayed/suppressed, or get more granular with the conditions only looking for cloudflare alerts:

 

Screen Shot 2019-07-02 at 5.08.02 PM.png

 

Once this Notification policy is configured, you will now be able to search for and enable it during the Maintenance window. Within a Maintenance policy, specify the timeframe  and the Notification policy.

Screen Shot 2019-07-02 at 5.11.56 PM.png

 

If you were to lean the other route of either enabling/disabling an integration and avoid any alert creation during this window - you'd simply want to adjust the filter of the Maintenance policy:

Screen Shot 2019-07-02 at 5.20.41 PM.png

You can also select the + Add rule button in the Maintenance policy to enable or disable multiple policies/integrations.

Best,

Nick

Daniel Leite
Contributor
November 25, 2019

While not very friendly way to do this, this do work!

it is not very friendly because  it is way too complex and requires multiple steps, most of our team do not know opsgenie well enough to even know how this setting work. But the worse is that requires a user with enough permissions to build and enable those settings, so at least it requires a escalation to the next level to be able to create those emergency maintenance settings.

 

A better option would be a user "silent/auto-ack all for..."  x minutes, next to the "ack all, close all"

It would be something that would not be used normally, but very useful to silent a alert spike, giving the team time to work in the problem and not in the never ending alert flood

 

maybe i need to create a feature request... (but how to do it ! ) :)

Like # people like this
Nick Haller
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
November 25, 2019

Hi Daniel - thanks for the feedback. You can alternatively suppress the notifications of an integration (found under Settings -> Configured integrations OR a team's Integration tab) - but will need to remember to turn this option back on after the fact:

 

suppress image.jpg

Carlos Reyes
I'm New Here
I'm New Here
Those new to the Atlassian Community have posted less than three times. Give them a warm welcome!
January 9, 2020

Is there a way to have a recurring Maintenance window?

 

We have some nightly tasks that generate alerts.. 

I cant find a way to have a nightly maintenance window say 21:00 to 21:15 

 

Any tips?

Like # people like this
Nick Haller
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
January 10, 2020

Unfortunately not at this time, but we do have an open feature request for this that I have added you to. Our tickets are not public, so I'll reach out directly with any updates/developments on the ticket. And if you'd like to contact support for an update yourself, the ticket for reference is ALX-818; Add recurring time windows as an option for Maintenance.

Thomas M.
I'm New Here
I'm New Here
Those new to the Atlassian Community have posted less than three times. Give them a warm welcome!
February 18, 2020

+1 for scheduled maintenance windows. Please include me on the updates.

Like # people like this
Nick Haller
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
February 18, 2020

Will do! No updates on this ticket yet, but I've added you to the ticket in case anything develops.

Nic.Zanelli
I'm New Here
I'm New Here
Those new to the Atlassian Community have posted less than three times. Give them a warm welcome!
March 6, 2020

Can I suppress notifications of an integration for only 1 entity?  We have 10 products that use the 1 integration - central monitoring tool - and we split them out by entity.  In a maintenance window scenario, I want to not have alerts created for 1 of the 10 products (entities).  I don't want to turn off alert creation for the other 9 products.

I can see how to suppress notifications by entity and I can see how to suppress alerts by integration, but I want to suppress alerts by entity.

Nick Haller
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
March 6, 2020

Hi Nic,

You'd want to configure a Notification Policy that suppresses the notifications for this entity/types of alerts.

Within the policy, filter on the field that the entity is parsed through (most likely the Entity) -> then have the Maintenance window enable this policy when either scheduled or started:

Entity ex1.jpgEntity ex2.jpg

Like Nic.Zanelli likes this
Kevin Six
I'm New Here
I'm New Here
Those new to the Atlassian Community have posted less than three times. Give them a warm welcome!
July 8, 2021

Please add me to the feature for planned recurring maintenance windows.

JiChung
I'm New Here
I'm New Here
Those new to the Atlassian Community have posted less than three times. Give them a warm welcome!
September 9, 2021

+1 please, as we would also like to suppress alerts for a specific service until certain time every day.

Christopher Emery
I'm New Here
I'm New Here
Those new to the Atlassian Community have posted less than three times. Give them a warm welcome!
November 24, 2021

+1 please add me to the recurring scheduled maintenance window feature request. This will be hugely beneficial to our usage of OpsGenie.

Darren Krohn
I'm New Here
I'm New Here
Those new to the Atlassian Community have posted less than three times. Give them a warm welcome!
January 20, 2022

@Nick Haller What is the status of this recurring scheduled maintenance item?  Will you please add me to the ticket if it helps as well? 

In my opinion, it'd be nice to see the Maintenance Policy feature allow you to enable/disable a Action Policy that is set up as well (it only lets you choose to Enable/Disable Notification Policies or Integrations).   For example, we have an Opsgenie  Action Policy that automatically tries to start back up an integrated tech item whenever Opsgenie receives an alert that our tech item is down.  During our maintenance windows, we purposely want the alerts to still occur, but we purposefully want to keep the tech item shut off for the multi-hour maintenance window, and we do not want our Action Policy to trigger.  I have made ways to work around this, but it'd be helpful to have.

Nick Haller
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
January 28, 2022

Hi @Darren Krohn ,

No update as of now on the recurring scheduled maintenance feature request. If/when we have an update, we'll reach out here! 

Like Nick McNeal likes this
0 votes
Bruno Araújo January 8, 2024

Is it possible to achieve this without using policies?

0 votes
Nick McNeal April 24, 2022

Wondering if this is the same issue my team has been trying to iron out for the last few weeks- we have an email integration that creates alerts for incoming emails from QRadar for one of our clients that we want to suppress during hours that we do not provide support for the client. Here's the notification policy that we created under the team that gets those alerts. Problem is, it doesn't appear to have any effect at all:

OG Notification Policy.png

Nick Haller
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
April 24, 2022

Hi @Nick McNeal ,

If this notification policy is consistent, and should always apply M-F 07:30-16:30, then I don't think enabling this via a maintenance policy is needed. You should be able to simply just keep the notification policy enabled at all times.

Regardless, what exactly is not working as expected? Is the notification policy not silencing these alerts during those days / hours? You may want to check your regex is correct as well. I typically use something like this: https://regex101.com/

It's also important to note that the hours defined in notification policies are applied to the account's timezone. So if your personal timezone varies, then that might be an issue: https://support.atlassian.com/opsgenie/docs/time-constraints-and-timezones/

Hope that helps. 

Nick Haller
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
April 24, 2022

Feel free to also open a ticket with our support team here: https://support.atlassian.com/opsgenie/

I noticed you blurred some of the above screenshot so if you needed to keep this private, we can take a look at things behind the scenes.

Like Nick McNeal likes this
Nick McNeal April 24, 2022

That was quick! Yeah, I originally had it set up without the maintenance policy- just getting desperate and trying different things at this point. The regex should work- it's using the same exact phrase that we use for the integration, which works fine (it's just the name of our client-hence the blurring). Basically, the problem is that we're still getting alerts during those "alert suppression" hours for alerts with those tags. The alerts are never suppressed during any hours for any time zone. It's like it just doesn't work. I have the right team, the right tag, the right hours, etc., and we're still getting alerts all hours of the day. 

Nick McNeal April 24, 2022

I've tried tags contains ClientName, as well. FWIW

Nick McNeal April 24, 2022

Thanks for the help, Nick! I've opened a support request, so that we can take a look at the issue behind the scenes with your support team. There's probably something I've overlooked in the setup, and another set of eyes should clear things up. 

Like Nick Haller likes this
Nick Haller
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
May 2, 2022

Hi @Nick McNeal ,

Following up here quick to share some insight into your use case with the Community. 

Basically Nick wanted to suppress notifications AND outgoing requests during off-hours for their Jira instance - so no issues are created during certain hours.

Notification policies can only suppress notifications sent to users, and cannot stop outgoing requests to Jira, or other integrations.

As a workaround we configured an alert policy under the team receiving these alerts that was restricted to time intervals, and was stripping the dynamic responder field from the alert - which essentially removes any responders from the alert entirely:

js1.jpg

js2.jpg

 

This keeps the team receiving the alerts as the Owner team on the alert, but removes them as a responder. That stops notifications from being sent to the team, and outgoing requests to other integrations:

js3.jpg

Ezra Van Den Bosch May 16, 2024

Why can't it be more user friendly? each change we make affects other alerts and vice versa

Ezra Van Den Bosch May 16, 2024

We only want to disable an integration for the specific alert - so alerts don't go through to MS Teams for instance. 

We don't want to disable the complete integration for a set period of time.

Ezra Van Den Bosch May 16, 2024

I am trying it with the alert policy now removing the dynamic responder field - will see if that works as expected

Ezra Van Den Bosch May 17, 2024

For my use case the following seems to fit:
1) alert policy that removes the responder field based on condition
2) notification policy that auto-closes the ticket based on the same condition

In this case the alert seems to be completely "silenced"

Ezra Van Den Bosch May 17, 2024

Only downside is that we do still receive an email when the ticket is closed

Ezra Van Den Bosch May 17, 2024

no email when it's set to auto-close though so that's good news!

Suggest an answer

Log in or Sign up to answer
TAGS
AUG Leaders

Atlassian Community Events