Heartbeat monitoring of CloudWatch integration

Question

Hi,

I've configured a CloudWatch integration so that CloudWatch alarms are delivered to OpsGenie. I have a CloudWatch alarm sending alarm events to SNS topic and I've configured OpsGenie CloudWatch integration endpoint as an HTTPS subscriber.

Now I would like to configure a constant monitoring of this integration to ensure that it works especially at the moment where a severe incident happens and CloudWatch alarm needs to be delivered to OpsGenie so somebody is paged.

My idea was to configure a "cron" task to publish a "ping" message event periodically on the same SNS topic so it's delivered to OpsGenie as a heartbeat. I want OpsGenie to treat these events as heartbeats and page me when a heartbeat message is not received for some longer period of time (as it might indicate that alarm delivery from AWS to OpsGenie doesn't work properly). Is it possible?

I know there is a feature called HeartBeat Monitoring. I thought that maybe I could create an heartbeat endpoint and configure it as another HTTPS subscriber on the same SNS topic. However, heartbeat endpoint uses a different "protocol" - it requires HTTP authorisation header (whereas CloudWatch integration includes the API key in the URL).

Am I missing something? How can I make sure that the alarm delivery works and if it stops working I get paged by OpsGenie?

Best regards,

Piotr

Piotr Bzdyl · Answer

Hi @Nick Haller,

Any update on my issue? Is there any other way to accomplish this? Or a plan in OpsGenie to implement a new feature supporting this?

Best regards,

Piotr

Samer Shami · Answer

Hi. Sorry to bring up an old thread. I am interested in finding a solution to this.

I need to alert when something doesn't appear in a SNS topic for a certain period of time. For example, if I am using AWS Backup and the SNS topic doesn't receive either a success or failure message for a certain amount of time, then OpsGenie should generate an alert.

So Heartbeat is perfect for this. However, I can't use the API version of Heartbeat because I can't send a header in the SNS HTTPS endpoint - nor can I use the email since Opsgenie does not confirm the email subscription.

So is there a way to get Heartbeats working from SNS? Alternatively, how do I alert on an integration not receiving anything for a certain period of time?

Piotr Bzdyl · Answer

Hi @Nick Haller ,

I finally got to test your suggestions to use OpsGenie Heartbeat email for an email subscription on the same SNS topic as the CloudWatch OpsGenie integration

Unfortunately, this won't work as Heartbeat email address doesn't give me access to the contents of the emails retrieved and I cannot even confirm SNS email subscription.

What's more, this still would be a suboptimal solution as I wouldn't be testing/monitoring 100% of the CloudWatch integration path as I would be monitoring CloudWatch alarm -> SNS topic -> email subscription -> OpsGenie Heartbeat path, and my goal is to monitor CloudWatch alarm -> SNS topic -> HTTPS subscription -> OpsGenie CloudWatch integration.

Thus I think this is a lacking feature in OpsGenie CloudWatch integration - it would be ideal to have this integration to support also "Heartbeat" feature directly.

Best regards,

Piotr