Come for the products,
stay for the community

The Atlassian Community can help you and your team get more value out of Atlassian products and practices.

Atlassian Community about banner
4,299,254
Community Members
 
Community Events
165
Community Groups

How to temporaly silent opsgenie notifications?

With the recent cloudflare network problem, our sites went down. Of course, opsgenie received alerts form several systems and kept beeping and showing up notifications  for all of then.

So the question is how to silent all notifications for X minutes, when we know already about the problem or a service is flapping.

They should still show in the site/app, but just do not show notification nor alerts... and of course, after the X minuted, any new alert should send notification again

 

Thanks for the help

2 answers

1 accepted

1 vote
Answer accepted
Nick H Atlassian Team Jul 02, 2019

Hi Daniel,

Thanks for reaching out. I think your best bet for this scenario would be a Maintenance Policy which allows you to enable or disable multiple policies/integrations with a single entry during a specified window.

If you would like the alerts created in your Opsgenie account, but notifications delayed/suppressed - you'll want this Maintenance window to enable a Notification Policy. If you would like the alerts to NOT be created you will want to disable an integration - obviously if an alert isn't created, no user(s) will be notified.

Both the Notification and Maintenance policies can be set on the team level under the Policies tab. If you were to lean towards enabling/disabling a Notification policy, you will first want to create this.

Within the filters of the Notification policy - you can select ALL alerts to be delayed/suppressed, or get more granular with the conditions only looking for cloudflare alerts:

 

Screen Shot 2019-07-02 at 5.08.02 PM.png

 

Once this Notification policy is configured, you will now be able to search for and enable it during the Maintenance window. Within a Maintenance policy, specify the timeframe  and the Notification policy.

Screen Shot 2019-07-02 at 5.11.56 PM.png

 

If you were to lean the other route of either enabling/disabling an integration and avoid any alert creation during this window - you'd simply want to adjust the filter of the Maintenance policy:

Screen Shot 2019-07-02 at 5.20.41 PM.png

You can also select the + Add rule button in the Maintenance policy to enable or disable multiple policies/integrations.

Best,

Nick

While not very friendly way to do this, this do work!

it is not very friendly because  it is way too complex and requires multiple steps, most of our team do not know opsgenie well enough to even know how this setting work. But the worse is that requires a user with enough permissions to build and enable those settings, so at least it requires a escalation to the next level to be able to create those emergency maintenance settings.

 

A better option would be a user "silent/auto-ack all for..."  x minutes, next to the "ack all, close all"

It would be something that would not be used normally, but very useful to silent a alert spike, giving the team time to work in the problem and not in the never ending alert flood

 

maybe i need to create a feature request... (but how to do it ! ) :)

Like Raghu Udiyar likes this
Nick H Atlassian Team Nov 25, 2019

Hi Daniel - thanks for the feedback. You can alternatively suppress the notifications of an integration (found under Settings -> Configured integrations OR a team's Integration tab) - but will need to remember to turn this option back on after the fact:

 

suppress image.jpg

Is there a way to have a recurring Maintenance window?

 

We have some nightly tasks that generate alerts.. 

I cant find a way to have a nightly maintenance window say 21:00 to 21:15 

 

Any tips?

Like # people like this
Nick H Atlassian Team Jan 10, 2020

Unfortunately not at this time, but we do have an open feature request for this that I have added you to. Our tickets are not public, so I'll reach out directly with any updates/developments on the ticket. And if you'd like to contact support for an update yourself, the ticket for reference is ALX-818; Add recurring time windows as an option for Maintenance.

+1 for scheduled maintenance windows. Please include me on the updates.

Like # people like this
Nick H Atlassian Team Feb 18, 2020

Will do! No updates on this ticket yet, but I've added you to the ticket in case anything develops.

Can I suppress notifications of an integration for only 1 entity?  We have 10 products that use the 1 integration - central monitoring tool - and we split them out by entity.  In a maintenance window scenario, I want to not have alerts created for 1 of the 10 products (entities).  I don't want to turn off alert creation for the other 9 products.

I can see how to suppress notifications by entity and I can see how to suppress alerts by integration, but I want to suppress alerts by entity.

Nick H Atlassian Team Mar 06, 2020

Hi Nic,

You'd want to configure a Notification Policy that suppresses the notifications for this entity/types of alerts.

Within the policy, filter on the field that the entity is parsed through (most likely the Entity) -> then have the Maintenance window enable this policy when either scheduled or started:

Entity ex1.jpgEntity ex2.jpg

Like Nic.Zanelli likes this

Please add me to the feature for planned recurring maintenance windows.

+1 please, as we would also like to suppress alerts for a specific service until certain time every day.

+1 please add me to the recurring scheduled maintenance window feature request. This will be hugely beneficial to our usage of OpsGenie.

@Nick H What is the status of this recurring scheduled maintenance item?  Will you please add me to the ticket if it helps as well? 

In my opinion, it'd be nice to see the Maintenance Policy feature allow you to enable/disable a Action Policy that is set up as well (it only lets you choose to Enable/Disable Notification Policies or Integrations).   For example, we have an Opsgenie  Action Policy that automatically tries to start back up an integrated tech item whenever Opsgenie receives an alert that our tech item is down.  During our maintenance windows, we purposely want the alerts to still occur, but we purposefully want to keep the tech item shut off for the multi-hour maintenance window, and we do not want our Action Policy to trigger.  I have made ways to work around this, but it'd be helpful to have.

Nick H Atlassian Team Jan 28, 2022

Hi @Darren Krohn ,

No update as of now on the recurring scheduled maintenance feature request. If/when we have an update, we'll reach out here! 

Like Nick McNeal likes this

Wondering if this is the same issue my team has been trying to iron out for the last few weeks- we have an email integration that creates alerts for incoming emails from QRadar for one of our clients that we want to suppress during hours that we do not provide support for the client. Here's the notification policy that we created under the team that gets those alerts. Problem is, it doesn't appear to have any effect at all:

OG Notification Policy.png

Nick H Atlassian Team Apr 24, 2022

Hi @Nick McNeal ,

If this notification policy is consistent, and should always apply M-F 07:30-16:30, then I don't think enabling this via a maintenance policy is needed. You should be able to simply just keep the notification policy enabled at all times.

Regardless, what exactly is not working as expected? Is the notification policy not silencing these alerts during those days / hours? You may want to check your regex is correct as well. I typically use something like this: https://regex101.com/

It's also important to note that the hours defined in notification policies are applied to the account's timezone. So if your personal timezone varies, then that might be an issue: https://support.atlassian.com/opsgenie/docs/time-constraints-and-timezones/

Hope that helps. 

Nick H Atlassian Team Apr 24, 2022

Feel free to also open a ticket with our support team here: https://support.atlassian.com/opsgenie/

I noticed you blurred some of the above screenshot so if you needed to keep this private, we can take a look at things behind the scenes.

Like Nick McNeal likes this

That was quick! Yeah, I originally had it set up without the maintenance policy- just getting desperate and trying different things at this point. The regex should work- it's using the same exact phrase that we use for the integration, which works fine (it's just the name of our client-hence the blurring). Basically, the problem is that we're still getting alerts during those "alert suppression" hours for alerts with those tags. The alerts are never suppressed during any hours for any time zone. It's like it just doesn't work. I have the right team, the right tag, the right hours, etc., and we're still getting alerts all hours of the day. 

I've tried tags contains ClientName, as well. FWIW

Thanks for the help, Nick! I've opened a support request, so that we can take a look at the issue behind the scenes with your support team. There's probably something I've overlooked in the setup, and another set of eyes should clear things up. 

Like Nick H likes this
Nick H Atlassian Team May 02, 2022

Hi @Nick McNeal ,

Following up here quick to share some insight into your use case with the Community. 

Basically Nick wanted to suppress notifications AND outgoing requests during off-hours for their Jira instance - so no issues are created during certain hours.

Notification policies can only suppress notifications sent to users, and cannot stop outgoing requests to Jira, or other integrations.

As a workaround we configured an alert policy under the team receiving these alerts that was restricted to time intervals, and was stripping the dynamic responder field from the alert - which essentially removes any responders from the alert entirely:

js1.jpg

js2.jpg

 

This keeps the team receiving the alerts as the Owner team on the alert, but removes them as a responder. That stops notifications from being sent to the team, and outgoing requests to other integrations:

js3.jpg

Suggest an answer

Log in or Sign up to answer
TAGS
Community showcase
Published in Confluence

An update on Confluence Cloud customer feedback – June 2022

Hi everyone, We’re always looking at how to improve Confluence and customer feedback plays an important role in making sure we're investing in the areas that will bring the most value to the most c...

264 views 2 5
Read article

Community Events

Connect with like-minded Atlassian users at free events near you!

Find an event

Connect with like-minded Atlassian users at free events near you!

Unfortunately there are no Community Events near you at the moment.

Host an event

You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events

Events near you