Our Prometheus (version 2.23.0) environment has alerting rules for monitored Kubernetes clusters for which Alertmanager sends alerts to Opsgenie. What we’re seeing is that Opsgenie receives six copies of the alert within a few milliseconds of each other, but none of them contain all of the fields. Several may have the description, or the responders, or priority, or tags, but none has a complete set of the relevant fields. This results in Opsgenie constructing an alert based on whichever copy it processes first, and counting the others as duplicates. Because the alert is always created from an incomplete copy of the alert, it is missing some of the information. I have had alertmanager send the alerts to a webhook to capture the JSON and it confirms the multiple copies that Opsgenie’s logs show. I have experimented with several different settings to try to get alertmanager to consolidate these alert notifications into one copy but haven’t had any success.
Now, I believe that https://github.com/prometheus/alertmanager/pull/2519 resolves the issue, but I can't roll it out until the next version of alertmanager comes out. Can anyone tell by my description if these are the same? Thanks! Doug -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/57e90032-d6f0-49d3-a49c-90a6b79d4d5en%40googlegroups.com.
