What Chris says: Prometheus will send a resolved alert to Alertmanager. If Alertmanager didn't receive any firing alert for the group, it won't send the resolved notification. You may get more insights by running Alertmanager with --log.level=debug.
On Tue, Feb 18, 2020 at 8:07 PM Chris Siebenmann <cks.prom-users...@cs.toronto.edu> wrote: > > > If I understand correctly, prometheus doesn't send any "resolved" > > message to alertmanager: it just stops sending alerts. Alertmanager > > treats the lack of alert as meaning "resolved". > > > > Therefore, if you receive the "resolved" message, then this proves > > that alertmanager must have received the original alert. > > > > Hence you need to check the alertmanager logs to determine why it > > couldn't send out the original alert E-mail. > > Modern versions of Prometheus do send specific 'resolved' messages to > Alertmanager when an alert ends. I don't know what Alertmanager does > if it receives a 'resolved' message for an alert that it hasn't seen > before, though. > > If Prometheus stops periodically sending information about current > alerts to Alertmanager (for example because it's been stopped or has > crashed), the current alerts will normally time out in a few minutes and > be declared as resolved by Alertmanager. But normally this shouldn't > happen; Prometheus re-notifies Alertmanager about any current alerts > on a regular basis. > > - cks > > -- > You received this message because you are subscribed to the Google Groups > "Prometheus Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to prometheus-users+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-users/20200218190718.AAC183201A1%40apps1.cs.toronto.edu. > -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/CAM6RFu5S5OtU%2B1jKvHMrM9-0Fgb3E0BkF3CGGcgtVt8%3D6S6GwA%40mail.gmail.com.
