As has been mentioned on this list recently an alert being resolved is due to the alert stopping being sent. There is no specific "alert resolution" message being sent.
The Alertmanager sends the resolution alert and only has access to the data at the time the alert was still firing, so has no idea about the current value. The recommend would be to adjust the wording on the alert to emphasise that the value is from when it was active. On 17 May 2020 11:06:46 BST, Rajesh Reddy Nachireddi <[email protected]> wrote: >Thanks Julius, > >we are using the predictive analytics for disk usage and only issue is >with >resolved alerts doesn't have the current utilisation. > >How to handle this? > >and also above git link looks too old w.r.t syntax, could you give us >latest one which has YAML format. > >Thanks, > >Rajesh > > > >On Sun, May 17, 2020 at 3:25 PM Julius Volz <[email protected]> >wrote: > >> In this case you can just directly refer to the sample value of the >time >> series returned by your expression (since it's already percentage you >> want), something like this: >> >> description: "Filesystem at {{$labels.mountpoint}} on >> {{$labels.instance}} is only {{$value}}% free." >> >> See also >> >https://prometheus.io/docs/prometheus/latest/configuration/alerting_rules/#templating >> >> For other situations you can also really run another query from the >> annotation template, see >> >https://prometheus.io/docs/prometheus/latest/configuration/template_examples/#simple-iteration >> >> Btw. you might want to look at node_filesystem_avail_bytes rather >> than node_filesystem_free_bytes, as free bytes also includes space >reserved >> for the root user, but you may run into problems with other users' >disk >> usage before that reserved space runs full. >> >> Btw. another thing: depending on the use case, consider looking into >> predictive disk usage alerts rather than alerts with static limits: >> https://www.robustperception.io/reduce-noise-from-disk-space-alerts >> >> On Sun, May 17, 2020 at 11:21 AM Rajesh Reddy Nachireddi < >> [email protected]> wrote: >> >>> Hi, >>> >>> How to use the query expression as part of annotations in alert rule >>> >>> - alert: OutOfDiskSpace >>> expr: node_filesystem_free_bytes / node_filesystem_size_bytes * >100 < 10 >>> for: 1m >>> labels: >>> severity: Critical >>> annotations: >>> description: "Disk has occupied the <exprsession> percentage >>> >>> Regards, >>> >>> Rajesh >>> >>> >>> >>> -- >>> You received this message because you are subscribed to the Google >Groups >>> "Prometheus Users" group. >>> To unsubscribe from this group and stop receiving emails from it, >send an >>> email to [email protected]. >>> To view this discussion on the web visit >>> >https://groups.google.com/d/msgid/prometheus-users/CAEyhnpLmr%2BCwB1GfBS0-3vLV2Zb3LuFnL7pH5McOfGyWbSwi0Q%40mail.gmail.com >>> ><https://groups.google.com/d/msgid/prometheus-users/CAEyhnpLmr%2BCwB1GfBS0-3vLV2Zb3LuFnL7pH5McOfGyWbSwi0Q%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> >> >> >> -- >> Julius Volz >> PromLabs - promlabs.com >> > >-- >You received this message because you are subscribed to the Google >Groups "Prometheus Users" group. >To unsubscribe from this group and stop receiving emails from it, send >an email to [email protected]. >To view this discussion on the web visit >https://groups.google.com/d/msgid/prometheus-users/CAEyhnpJJjZ3QrymhgAW_UawCKEDbq9z%2B3oHECu7Zig6PtzzDCg%40mail.gmail.com. -- Sent from my Android device with K-9 Mail. Please excuse my brevity. -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/77B147B9-79B6-4ED9-A83F-6B9F2F44079A%40Jahingo.com.
