nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Opsgenie alert aggregation

Madani_g84vg1

We are currently using opsgenie integrated with logstash, and now we have a need to have the following scenarios:

- if we encountered 10 times the same error in less than 10mn trigger an alert

I would like to know if there is a way to do it on opsgenie, as currently we are creating the alert by doing customization on logstash. at each time we have an error the alert is created on opsgenie, but now for the same alert, we want to have the logic to say if we got 5 times the same alert in less than 10 mn we trigger an alarm.

Thank you for your support

Accepted answers

Shashwat Khare

Hi @Madani ,

Yes, this can be achieved in Opsgenie, however, the alert creation can't be customized based on the iteration frequency in a given time interval.

Instead, you can control the notifications to be sent/delayed until an alert occurs 10 times an interval of 10 minutes coming from Logstash to Opsgenie.

Herein, the alias value in Opsgenie will be used to determine which iteration of an alert is recurring to delay the notifications until it happens 10 times in 10 minutes.

Here's an example screenshot of how this notification policy would look like for an Opsgenie team:
Screenshot 2024-10-07 at 2.18.09 PM.png

Best,
Shashwat

All comments

Shashwat Khare

Madani_g84vg1

Thank You, i was finally using the plugin throttle in my pipeline but finally will go with this solution which it easier. Last question i am currently wanted even to be alerted if nothing happened in the log file during more than 1 hour, was thinking of changing my pipeline using heartbeat and the pluggin throttle, or there is an other easy way to handle it with opsgenie ? thank you Shashwat

Shashwat Khare

Hello @Madani ,

Yes, using an Opsgenie heartbeat would be the best solution wherein you can configure the heartbeat period anywhere between 30 minutes to 1 hour as in the below example:
Screenshot 2024-10-07 at 5.32.03 PM.png

Best,
Shashwat

Madani_g84vg1

yes but the issue is how to integrate it with Logstash i add the following in logstash :

# Input pour les heartbeats
heartbeat {
interval => 300 # Intervalle de 300 secondes (5 minutes)
type => "heartbeat" # Type d'événement pour identifier les heartbeats
}

in filter i add this :

if [type] != "heartbeat" {
mutate {
add_tag => ["log_detected"]
}
}

# Throttle to detect if no logs are received in the last hour
if [type] == "heartbeat" {
throttle {
before_count => 1 # After receiving an event, we check
period => 3600 # Period of 1 hour (3600 seconds)
key => "log_detected" # Monitor the received logs (non-heartbeat)
add_tag => ["no_logs_last_hour"] # Add a tag if no log has been detected
}
}

# If no log has been detected for the last hour, trigger an alert
if "no_logs_last_hour" in [tags] {
mutate {
add_field => {
"opsgenieAction" => "create"
"alias" => "NoLogsInLastHour"
"description" => "No logs have been received in the last hour."
}
add_tag => ["opsgenie-alert"]
}
}

and in output i used this :

if "opsgenie-alert" in [tags] {
opsgenie {
apiKey => "your_api_key"
}
}

My question is do i need to do in this way, which means using throttle, because i dont see with just heartbeat and not using pluging throttle how it can works ? there is no documentation of logstash integration with opsgenie heartbeat. Thank you for your support