Opsgenie Integration with Prometheus Alertmanager

[Eugene Sheyko]

Hi Team,

We're trying to integrate our Prometheus Alertmanager with Opsgenie in Jira.

Can you please help how to get the additional fields like Description, environment and Summary be visible in the alert?

---

receivers:
  - name: default
    email_configs:
      - to: '{{ .GroupLabels.support_contacts }}'
        send_resolved: true

  - name: opsgenie_receiver
    opsgenie_configs:
      - api_key: <api_key>
        send_resolved: true
        description: '{{ .CommonAnnotations.description }}'
        details:
          alertname: '{{ .GroupLabels.alertname }}'
          instance: '{{ .GroupLabels.instance }}'
          severity: '{{ .CommonLabels.severity }}'
          priority: '{{ if eq .CommonLabels.severity "critical" }}P1{{ else }}P3{{ end }}'
        custom_fields:
          summary: '{{ .CommonAnnotations.summary }}'
          env: '{{ .GroupLabels.env }}'
          support_contacts: '{{ .GroupLabels.support_contacts }}'

time_intervals:
  - name: weekdays
    time_intervals:
      - weekdays: ['monday:friday']
        times:
          - start_time: '08:00'
            end_time: '24:00'

---

For example, we have the following Alertmanager rule set up.

---

- alert: WindowsServerCpuUsage
  expr: 100 - (avg by (instance) (rate(windows_cpu_time_total{mode="idle"}[2m])) * 100) > 90
  for: 0m
  labels:
    severity: warning
    team: infra
  annotations:
    summary: "Windows Server CPU Usage (instance {{ $labels.instance }})"
    description: |
      CPU Usage is more than 80%
      VALUE = {{ $value }}
      LABELS = {{ $labels }}

---

We receive the alerts and get them populated into Jira using Opsgenie in the following format as shown below.

[Prometheus]: [FIRING:1] <server> (WindowsServerCpuUsage warning infra)

Description

Windows Server CPU Usage (instance <server>)

The extra properties only contain the following values.

Extra properties
severity     warning
instance     <server>
alertname  
team          infra

Can you please advise if the additional fields (such as Description, environment and Summary) need to be also configured in Administration part of the UI and how to get them populated under the "Extra properties" section?

Thanks,
Eugene

[Rafael Meira]

Hey @Eugene Sheyko 

To get additional fields like "Description", "Environment", and "Summary" visible in Opsgenie alerts from Prometheus Alertmanager, you can try to follow these steps:


Your current configuration looks good. Make sure the alert rule in Prometheus has the necessary annotations and labels, like summary and description. You've already set these up in your alert rule.

Opsgenie:

Ensure that the Opsgenie integration is correctly set up with the API key and that it's configured to recognize these fields. Usually, Opsgenie will automatically map these fields if they're present in the incoming alerts.

Reload Alertmanager:

After making any changes, reload Alertmanager to apply the new configuration. You can do this by sending a SIGHUP signal or using the /reload API endpoint.

Verify in Opsgenie:

Once alerts come through, check Opsgenie to ensure the extra properties are showing up as expected. If not, there might be an issue with the integration setup or connectivity.

If you follow these steps, you should see those extra fields in Opsgenie. If they're still missing, double-check your configurations and logs for any errors.

https://support.atlassian.com/opsgenie/docs/integrate-opsgenie-with-prometheus/ 

https://prometheus.io/docs/alerting/latest/configuration/#opsgenie-receiver-%3Copsgenie_config%3E 

I would also advise if you could open a support ticket so we can better investigate here:


https://support.atlassian.com/opsgenie/ -> Contact Support

[Eugene Sheyko]

Hi @Rafael Meira,

Thank you for your response.

Could it be related to notification policies in the Teams that need to be set up behind the scenes? As the default Properties are visible there in the Prometheus alert ingested into Opsgenie, in JSM.

At this point, the only question is how to get the other additional fields preserved and make them available in all the alerts if they also meet the criteria of the Prometheus alert.

Please find the screenshots attached if this could be helpful.

[Rafael Meira]

@Eugene Sheyko 

To make sure fields like Description, Environment, and Summary show up in Opsgenie alerts under "Extra Properties," you can attempt to follow these steps:

Update Alertmanager Config: Your current config looks good. Ensure fields like description, summary, and env are included:

details:
alertname: '{{ .GroupLabels.alertname }}'
instance: '{{ .GroupLabels.instance }}'
severity: '{{ .CommonLabels.severity }}'
custom_fields:
summary: '{{ .CommonAnnotations.summary }}'
env: '{{ .GroupLabels.env }}'

Verify that your Prometheus alert rules include these fields. For example:

annotations:
summary: "CPU Usage High"
description: "Instance {{ $labels.instance }} exceeded CPU usage"
labels:
env: production

Go to Teams > Integrations, edit the Prometheus integration, and confirm fields like summary and description are mapped correctly.

Apply the changes by reloading Alertmanager (SIGHUP or /reload API).

Check if the fields appear in the "Extra Properties" section. If not, confirm Prometheus is sending the fields and your Opsgenie setup isn’t filtering them out.

Besides that, to assist you further, you would need to create a support ticket with us, so we can see any errors, investigate, get our hands into the instance and so on.

hope this could help.
 

[Eugene Sheyko]

Thank you for your response, I believe only user with the Administrator would be able to see the Teams > Integrations functionality? We will check this and will confirm soon.

Could you please advise what is the proper way to create a support ticket with you?

Thanks,
Eugene