N-Central to OpsGenie alerts

[Michael Hodgson]

Hi,

We have Opsgenie as part of the JSM standard package (cloud), and looking to collate our alerts/monitoring in one location.

I have set up the integration as per the article here:

https://support.atlassian.com/opsgenie/docs/integrate-opsgenie-with-solarwinds-msp-n-central/

At this point I'm not looking for anything too fancy, I'm just essentially after:

- An alert is triggered in N-Central

- "Alert" is then created in OpsGenie

- Alert is acknowledged in N-Central

- Alert is acknowledged In OpsGenie

- Alert is resolved (returns to "normal") in N-Central

- Alert is resolved/closed In OpsGenie

So essentially replicating what it's currently doing, purely so we can capture the alert trigger, acknowledgement and resolve in one location.

So our set up/testing:

- Alert is created in Opsgenie (as expected)

- Alert is acknowledged in Opsgenie and feeds through the name of who acknowledged from N-Central which is great via a note

- However, when the alert is "resolved" or "returns to normal" - I can see this coming in the audit logs within Opsgenie with the old state = failed, and new state = normal, but this doesn't update the Opsgenie alert status or add a comment

I can then see in the audit logs the next interaction.

It essentially treats the "resolved" as an additional acknowledgement, and we're met with a message in the audit logs of:
"Can not execute [Acknowledge] action. Reason: Alert is already acknowledged"

Is this expected? or have I missed something? The integration for receiving notifications just looks like a simple API URL to the user, which works fine on a top level. 

I can see there's an OEC integration but this from my understanding is the 2-way sync from Opsgenie back to N-Central which I'm not really interested in doing right now.

Any help would be lovely, thank you! 

[Nick H]

Hi @Michael Hodgson ,

Important to note that all of the integration actions:

are processed in an order of operations from top >> down - starting with Opsgenie reviewing the the Ignore (if there are any configured), then the Create alert action(s), then the Close alert action(s), and so on. Also - Opsgenie can only match once and will execute the first matching action / filter.

If the log states, "Can not execute [Acknowledge] action. Reason: Alert is already acknowledged" then I have to assume the Close action did not match so Opsgenie moved onto the Ack action and did find a matching filter / condition.

Note the screenshot above is of the Advanced tab which is only included with the standalone Opsgenie Standard and Enterprise plans, or the JSM/O Premium and Enterprise plans.

So if possible, review your Close action's filter / conditions and the log to see where there might be a misconfiguration or disconnect. 

[Michael Hodgson]

Hi Nick,

Thanks for this - I appreciate your comments and understand what you've mentioned above. 

However, just on the "advanced tab" - as we are using Opsgenie as part of Jira service management standard, this isn't something available to us right now.

Based on the article I attached in my original post I was under the impression this would handle the requests accordingly so I'm not sure if there are any other options or things to investigate without that advanced tab.

I had hoped someone might have come across some obscure setting that I missed or configuration from Solarwinds N-Central in itself.

[Nick H]

Hi @Michael Hodgson ,

My screenshot above of the Close alert action is what is set by default on the N-Central integration - regardless of plan. So maybe check your log and those two fields to see whether or not they match.

[Michael Hodgson]

Thanks Nick, so I've performed a few tests today and if I'm reading your screenshot correctly, the close alert should trigger when:

- New state = Normal

- Acknowledgement = Empty

If this is the criteria, the latest example I have tested matched this.

Scenario:

- Alert had been triggered from N-Central

- The alert was not acknowledged in N-Central

- This triggered the alert creation in Opsgenie

- The alert was not acknowledged in Opsgenie

- The alert then had been resolved in N-Central (without acknowledgement)

- This sent the information to Opsgenie and looking at the logs (see attached raw log format in opsgenie)

- It does appear to be sending over the "new state" as Normal and there had been no acknowledgement from either side

- However, the alert within Opsgenie is still open - and just classes it as acknowledged

Please see the additional attachment for the ack note on resolve. As you can see there are no user data pull through, (whereas when ack in N-Central it pulls the users name), and this "note" as well as the acknowledgement action, was triggered on the Resolve from N-Central (as shown in the log)

(note the timings are out by an hour as our N-Central appears to be an hour out which we're looking to correct - not sure if this affects anything..  this is the correct notification and alert being referred to though - I confirmed this at the time using the alertTinyID)

[Nick H]

The two conditions are actually:

• New state = Normal
• Acknowledgement Time = Empty

Above in your screenshot, the Ack Time field is showing as [no data available in DB] - which I assume is an actual string? You can check the log by reviewing the Raw tab of it to be sure. 

[Michael Hodgson]

Hi Nick,

Performing a few additional checks, I can confirm that even on resolve and no acknowledgement from either Opsgenie or N-Central side, the "resolve" is coming in as an "acknowledgement alert" regardless

The logs, are similar to what I've sent previously. This is from the opsgenie logs and Raw format.

This is the log triggered after N-Central marked a "resolve" on it's side, and the data received seems appropriate e.g.

- QualitativeOldState = Failed

- QualitativeNewState = Normal

- The "no data available in DB" is what it sends across when no one has acknowledged from N-Central side

(If they do acknowledge on N-Central this data is of course, populated)

Based on this - this appears to meet that criteria you mentioned of new state = normal, and acknowledgement time = empty - as you can see in the screenshot