Hi, On 9/23/20 3:02 PM, iono sphere wrote: > This is working as expected. AlertManager calls > http://localhost:2222/alert and I got "alert received". But here's the > thing, it calls two times - as expected, but the first time is when the > alert goes off, but the second time is when the alert has been resolved. > I do not know which is for "the alert goes off" and "resolve". I have > checked in the req parameter, but I do not see anything that could be of > use, unless I am missing something... > > So: > > 1. how can we differentiate the "alert goes off" and the "alert has been > resolved" in this case?
The Webhook Payload contains a "status" field which should be set to "resolved" in these cases: https://prometheus.io/docs/alerting/latest/configuration/#webhook_config I'm not exactly sure what your example is (NodeJS?) and how the web handler works exactly. However, if "res" contains the other JSON fields, it should also contain the "status" field. If it does not, you might have to look at some other part of the request, maybe? > 2. What is the "resolve" time? Immediately if the expr in rules.yml is > false in the next evaluation_interval? Without having looked into the actual code: Prometheus only knows if an alert changed when the rule has been re-evaluated. This happens after evaluation_interval. Only then, it can notice that the alert is resolved and will notify Alertmanager which will then notify your webhook after several aggegrations/cleanups (e.g. group_interval/group_wait, etc.). So I would say: The notification is not immediate, but directly after the intended grace periods. Kind regards, Christian -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/ec028df8-6d80-7d7f-30ac-f8800aa80e5c%40hoffmann-christian.info.
