2 answers
1 accepted
Hello @Donny Gison
Hope your day is great,
As I can see from the screenshot shared you have just set the initial close alert rule.
So as mentioned in my previous response, you can set a "Close Alert" action which will close an existing Opsgenie open alert based on its "alias" value.
[The alias is the unique identifier for "open" alerts. There can only be one "open" alert with the same alias in Opsgenie.]
So for the Close Alert rule to work, the below conditions are required to match, and the setup will be as follows.
1. You will initially configure a "Create alert" action. A "Create Alert" action will create a new Opsgenie alert if the condition set of its filter matches the data. You need to make sure to set a custom "Alias" value for this "Create alert" action.
2. A "Close Alert" action will close an existing Opsgenie alert based on its "Alias" which means, for the "Close alert" action to work you need to have an Open alert with the same Alias field value configured in both "Create alert" and Close alert" action.
3. The actions are evaluated in order, each time the alert payload reaches Opsgenie it will be first evaluating the first action configured under your account.
Ex: You have created "Create alert -1", "Create alert -2", "Close alert -1" and "Close alert -2" actions under your integration settings.
Alert triggered from Dynatrace,
- It first checks the "Create alert -1" action if it matches alert will be created
- if not then It will go to the next action "Create alert -2" if this alert action is matching then based on its rules alert will be created. If there is already an alert open with the same information It will get de-duplicated.
- If not it will then route to "Close alert -1" at this point if this action matches then Opsgenie will check if there is any open alert with the "Alias" matching, and It will close that alert accordingly and It will not check any further with the last action "Close alert -2"
For any action set up for integration, an Alias value must be defined and matched exactly in order for the action to be executed.
Hope this information is helpful, please let us know if you have any further questions. I will be happy to assist.
Regards,
Mohammed Mubeen
Support Engineer, Atlassian
Hi @Mubeen Mohammed ,
That's helpful. Made me understand Actions more. May I please seek further feedback.
In your example the Alias to match is hard coded. What if it changes like DT problem number.
DT Alert Creation Message -
[PROD] SmartQ UW: OPEN Problem P-220343162
DT Alert Closure Message -
[PROD] SmartQ UW: RESOLVED Problem P-220343162
May Alias values is likely not right :)
Apologies :) I think I misunderstood the value for Alias. I have updated and trying the below .
Just waiting for the next alert to test it out. Opted to not use Create Alert, for now, to not mess with the count of alerts.
Unfortunately this did not work as well.
Won't the DT Alert for Resolved be logged somewhere as well? We seem to not be getting one. The closure stopped for us sometime Feb. Now I am wondering if its the Integration API. Not sure
Here is a sample log when the Auto Closure from a Resolved DT alert was still working when received, hope this helps.
The creation seems to work. We just had 2 DT alerts today that Opsgenie was able to tag duplicate.
We got 2 DT alert emails with different email subjects but the same body of text, Opsgenie was able to have them in one alert only, just increased the count to 2.
Creation works including duplicates but closure doesn't. I still don't fully get Alias haha
Here's the Create set-up. Maybe I need to define exactly the same fields for Close alert?
Close just have Alias, User and Notes.
Help will be appreciated.
Hello @Donny Gison
I can help explain the concept of "Alias", as mentioned earlier The Alias field is a user-defined unique identifier for "open" alerts.
From the last deduplication message screenshot you sent, you can see it has the Message: "[Dynatrace]: [PROD]: SmartQ: U/W ................... low disk space"
And you can see the count increased to 2. You got this as "Alias" because you have set the "Create alert" action alias field as "[Dynatrace]: {{message}}".
Since the alert was received with the same alias value and it matched with the "Create alert" action so it got deduplicated.
Please note: As mentioned earlier in my previous response,
The payload received from Dynatrace will be checked in the order your alert actions are created.
As per the order, the first section is "Create alert" when the alert payload comes and it matched with the first action "Create alert" it will create the alert. So you need to send a different payload that matches only with "close alert" to be able to close the alert which is already created with the same "Alias" value.
Incoming payload from Dynatrace > -- matches with -- > "Create alert" = Alert will be created.
Incoming payload from Dynatrace >-- matches with -- > "Create alert" and an alert is already open = It will be deduplicated
Incoming payload from Dynatrace >-- matches with -- > "Close alert" and an alert with the same alias value is already in open state = Alert will be closed.
For your testing try to set the "alias" values in both create alert and close alert action as I specified in the previous response screenshots.
Hope this answers your queries.
Regards,
Mohammed Mubeen
Support Engineer, Atlassian
Thanks for continuing to respond. I am slow in understanding Aliases I think. Let me try the below.
- Create Alert
- In the sample deduplication, the Message are not actually identical.
1st DT Alert has “Asia ESB”
2nd DT alert that became deduplicated has “SmartQ UW”
Opsgenie was successful to create and deduplicate even with some words different on the description of the 2 DT alerts.
Now, for Close alert, DT sent these
For the first alert - different is just the word "RESOLVED" instead of "OPEN"
For the second alert - different is just the word "RESOLVED" instead of "OPEN"
This confused me on how Alias works. Deduplicate was able to work with different words on the Description of the 2 DT alerts received. But Close was not able to with just "Resolved" as the difference. I might be understanding Alias incorrectly.
We tried adding a filter of looking for "Resolved" in the Close alert because the default of just Alias someshow stopped working in February.
Maybe given the DT alerts we receive you can be very specific on what may work as Close alert set-up? To make it work again. And other Action set-up I need to do based on what we get above.
Also, is there a way to confirm that the "Resolved" alert from DT did reach Opsgenie to go through the Actions? This is one thing I can't find so far.
Hope you continue to help me solve this. Thank you.
Hello @Donny Gison
No worries I can help clarify your queries with Alias.
For the alerts which got deduplicated with the Alias field and from the screenshot I see you have just set "{{alias}}". To help understand how you got two different alerts message content was de-duplicated, I have provided the information in my previous responses which is a common setup for most of the Opsgenie configuration.
But your recent query requires checking configuration and logs within from your Opsgenie instance because it will be a unique setup/configuration for individual instances. You can reach out to me at https://support.atlassian.com/contact/#/ so I can further check account-specific details to help further.
Also, I would like to share a specific example configuration that will allow you to create and close an alert, so you can test this further to understand/implement it accordingly.
For our example, let's just consider you are getting the following data for an instance state change and you need to set Create and Close actions accordingly.
Ex:
Server.abc is currently not responding. (Create alert)
The issue with Server.abc is resolved now. (Close alert)
Initial setup
As per this setup when I receive the payload as "Server.abc is currently not responding" it will create an alert for me as I have set a message containing "not responding" as a condition to get the alert to create.
Now the "close alert" configuration setup in relation to the alert created by the above "Create alert" condition.
As you can see from the screenshot I have set the condition as a message that contains "resolved now" should match and with the same alias value if there is any open alert then close it.
Hope this clarified the Alias role and alert conditions. You can try to test with the exact value to understand how it works.
Just want to add another payload example in relation to the examples shared above to clear your understanding.
Let's say you have sent the resolved payload a little differently as follows "The server.abc which was not responding that is resolved now"
This technically resolves the alert as it has "resolved now" in it, but it will not instead it will de-duplicate the alert. Because before the alert condition match the "Close alert" action it matched with the "Create alert" action "not responding" hence it de-duplicated.
I have specified in my previous responses Payload checks the alert action in order, the first action it matches will be executed and it will stop from there.
Hope this example clarified your query with "Alias".
Regards,
Mohammed Mubeen
Support Engineer, Atlassian
Hi @Mubeen Mohammed , I raised
https://getsupport.atlassian.com/servicedesk/customer/portal/45/OGSP-86337
to investigate the logs and confirm that DT Resolved alerts do reach our Opsgenie instance.
Tried defining an Alias of [PROD] for both Create and Close, unfortunately it did not work.
The DT alerts
For Create
For Close, but seem to not consumed by Opsgenie
Hello @Donny Gison
Yes, I got the ticket you have created, I will further help with the ticket you have opened :)
Also to clarify on the screenshot you shared above for the "create alert" action rule you have mentioned the filter condition as "Match all alerts"
As mentioned in my previous responses all the Alert actions for CREATE or CLOSE will be checked in order, if the alert payload matches with the first one then the next alert action will not be checked.
Since you have set the alert filter as "Match all alerts" it will match with all alerts incoming payload, including the "Resolved" payload, hence no "Close Alert" actions were executed.
Instead of "Match all alerts" you can set a specific value as one of the alert conditions to test it again.
Regards,
Mohammed Mubeen
Support Engineer, Atlassian
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.