Create
cancel
Showing results for 
Search instead for 
Did you mean: 
Sign up Log in

N-Central to OpsGenie alerts

Michael Hodgson March 22, 2023

Hi,

We have Opsgenie as part of the JSM standard package (cloud), and looking to collate our alerts/monitoring in one location.

I have set up the integration as per the article here:

https://support.atlassian.com/opsgenie/docs/integrate-opsgenie-with-solarwinds-msp-n-central/

 

At this point I'm not looking for anything too fancy, I'm just essentially after:

- An alert is triggered in N-Central

- "Alert" is then created in OpsGenie

- Alert is acknowledged in N-Central

- Alert is acknowledged In OpsGenie

- Alert is resolved (returns to "normal") in N-Central

- Alert is resolved/closed In OpsGenie

So essentially replicating what it's currently doing, purely so we can capture the alert trigger, acknowledgement and resolve in one location.

 

So our set up/testing:

- Alert is created in Opsgenie (as expected)

- Alert is acknowledged in Opsgenie and feeds through the name of who acknowledged from N-Central which is great via a note

- However, when the alert is "resolved" or "returns to normal" - I can see this coming in the audit logs within Opsgenie with the old state = failed, and new state = normal, but this doesn't update the Opsgenie alert status or add a comment

I can then see in the audit logs the next interaction.

It essentially treats the "resolved" as an additional acknowledgement, and we're met with a message in the audit logs of:
"Can not execute [Acknowledge] action. Reason: Alert is already acknowledged"

 

Is this expected? or have I missed something? The integration for receiving notifications just looks like a simple API URL to the user, which works fine on a top level. 

I can see there's an OEC integration but this from my understanding is the 2-way sync from Opsgenie back to N-Central which I'm not really interested in doing right now.

Any help would be lovely, thank you! 

 

1 answer

0 votes
Nick H
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
March 22, 2023

Hi @Michael Hodgson ,

Important to note that all of the integration actions:

sw1.jpg

are processed in an order of operations from top >> down - starting with Opsgenie reviewing the the Ignore (if there are any configured), then the Create alert action(s), then the Close alert action(s), and so on. Also - Opsgenie can only match once and will execute the first matching action / filter.

 

If the log states, "Can not execute [Acknowledge] action. Reason: Alert is already acknowledged" then I have to assume the Close action did not match so Opsgenie moved onto the Ack action and did find a matching filter / condition.

Note the screenshot above is of the Advanced tab which is only included with the standalone Opsgenie Standard and Enterprise plans, or the JSM/O Premium and Enterprise plans.

So if possible, review your Close action's filter / conditions and the log to see where there might be a misconfiguration or disconnect. 

Michael Hodgson March 22, 2023

Hi Nick,

Thanks for this - I appreciate your comments and understand what you've mentioned above. 

However, just on the "advanced tab" - as we are using Opsgenie as part of Jira service management standard, this isn't something available to us right now.

Based on the article I attached in my original post I was under the impression this would handle the requests accordingly so I'm not sure if there are any other options or things to investigate without that advanced tab.

I had hoped someone might have come across some obscure setting that I missed or configuration from Solarwinds N-Central in itself.

Nick H
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
March 22, 2023

Hi @Michael Hodgson ,

My screenshot above of the Close alert action is what is set by default on the N-Central integration - regardless of plan. So maybe check your log and those two fields to see whether or not they match.

Michael Hodgson March 23, 2023

Thanks Nick, so I've performed a few tests today and if I'm reading your screenshot correctly, the close alert should trigger when:

- New state = Normal

- Acknowledgement = Empty

If this is the criteria, the latest example I have tested matched this.

 

Scenario:

- Alert had been triggered from N-Central

- The alert was not acknowledged in N-Central

- This triggered the alert creation in Opsgenie

- The alert was not acknowledged in Opsgenie

 

- The alert then had been resolved in N-Central (without acknowledgement)

- This sent the information to Opsgenie and looking at the logs (see attached raw log format in opsgenie)

- It does appear to be sending over the "new state" as Normal and there had been no acknowledgement from either side

- However, the alert within Opsgenie is still open - and just classes it as acknowledged

 

Please see the additional attachment for the ack note on resolve. As you can see there are no user data pull through, (whereas when ack in N-Central it pulls the users name), and this "note" as well as the acknowledgement action, was triggered on the Resolve from N-Central (as shown in the log)Raw log format on Opsgenie alert w-o acknowledgement.pngAck note on resolve.PNG

 

(note the timings are out by an hour as our N-Central appears to be an hour out which we're looking to correct - not sure if this affects anything..  this is the correct notification and alert being referred to though - I confirmed this at the time using the alertTinyID)

Nick H
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
March 23, 2023

The two conditions are actually:

  • New state = Normal
  • Acknowledgement Time = Empty

 

Above in your screenshot, the Ack Time field is showing as [no data available in DB] - which I assume is an actual string? You can check the log by reviewing the Raw tab of it to be sure. 

Michael Hodgson March 27, 2023

Hi Nick,

 

Performing a few additional checks, I can confirm that even on resolve and no acknowledgement from either Opsgenie or N-Central side, the "resolve" is coming in as an "acknowledgement alert" regardless

The logs, are similar to what I've sent previously. This is from the opsgenie logs and Raw format.

This is the log triggered after N-Central marked a "resolve" on it's side, and the data received seems appropriate e.g.

- QualitativeOldState = Failed

- QualitativeNewState = Normal

- The "no data available in DB" is what it sends across when no one has acknowledged from N-Central side

(If they do acknowledge on N-Central this data is of course, populated)

 

Based on this - this appears to meet that criteria you mentioned of new state = normal, and acknowledgement time = empty - as you can see in the screenshot

Ack logs on resolve.png

Suggest an answer

Log in or Sign up to answer
TAGS
AUG Leaders

Atlassian Community Events