Opsgenie alert aggregation

[Madani]

We are currently using opsgenie integrated with logstash, and now we have a need to have the following scenarios:

 - if we encountered 10 times the same error in less than 10mn trigger an alert

I would like to know if there is a way to do it on opsgenie, as currently we are creating the alert by doing customization on logstash. at each time we have an error the alert is created on opsgenie, but now for the same alert, we want to have the logic to say if we got 5 times the same alert in less than 10 mn we trigger an alarm.

Thank you for your support

[Shashwat Khare · Accepted solution]

Hi @Madani , 

Yes, this can be achieved in Opsgenie, however, the alert creation can't be customized based on the iteration frequency in a given time interval.

Instead, you can control the notifications to be sent/delayed until an alert occurs 10 times an interval of 10 minutes coming from Logstash to Opsgenie.

Herein, the alias value in Opsgenie will be used to determine which iteration of an alert is recurring to delay the notifications until it happens 10 times in 10 minutes.

Here's an example screenshot of how this notification policy would look like for an Opsgenie team:


Best,
Shashwat

[Madani]

Thank You, i was finally using the plugin throttle in my pipeline but finally will go with this solution which it easier. Last question i am currently wanted even to be alerted if nothing happened in the log file during more than 1 hour, was thinking of changing my pipeline using heartbeat and the pluggin throttle, or there is an other easy way to handle it with opsgenie ? thank you Shashwat

[Shashwat Khare]

Hello @Madani ,

Yes, using an Opsgenie heartbeat would be the best solution wherein you can configure the heartbeat period anywhere between 30 minutes to 1 hour as in the below example:


Best,
Shashwat

[Madani]

yes but the issue is how to integrate it with Logstash i add the following  in logstash :

# Input pour les heartbeats
heartbeat {
interval => 300 # Intervalle de 300 secondes (5 minutes)
type => "heartbeat" # Type d'événement pour identifier les heartbeats
}

in filter i add this :

if [type] != "heartbeat"  {
mutate {
add_tag => ["log_detected"]
}
}

# Throttle to detect if no logs are received in the last hour
if [type] == "heartbeat" {
throttle {
before_count => 1 # After receiving an event, we check
period => 3600 # Period of 1 hour (3600 seconds)
key => "log_detected" # Monitor the received logs (non-heartbeat)
add_tag => ["no_logs_last_hour"] # Add a tag if no log has been detected
}
}

# If no log has been detected for the last hour, trigger an alert
if "no_logs_last_hour" in [tags] {
mutate {
add_field => {
"opsgenieAction" => "create"
"alias" => "NoLogsInLastHour"
"description" => "No logs have been received in the last hour."
}
add_tag => ["opsgenie-alert"]
}
}

and in output i used this :

if "opsgenie-alert" in [tags] {
opsgenie {
apiKey => "your_api_key"
}
}

My question is do i need to do in this way, which means using throttle, because i dont see with just heartbeat and not using pluging throttle how it can works ? there is no documentation of logstash integration with opsgenie heartbeat. Thank you for your support