Thanks for advice,
So in this case I just need to use absent like this In alert?:
- alert: Resource group in cluster is down
expr: absent(windows_mscluster_resourcegroup_state {name!~"Available
Storage"}) == 1
for: 10s
labels:
severity: "[Cluster]"
annotations:
summary: "Resource group in cluster is down!"
description: "{{ humanize $value }}"
This one will send message, when metric is missing?
From: Matthias Rampke <matth...@prometheus.io>
Sent: Friday, December 9, 2022 8:57 AM
To: Sebastian Glock <sebagloc...@gmail.com>
Cc: Prometheus Users <prometheus-users@googlegroups.com>
Subject: Re: [prometheus-users] Null value in alerts
When you say "the value is missing", what condition exactly do you want to
alert on?
To detect that there is *no* metric matching your selector, you can use the
absent(…) function. It returns 1 when … is nothing.
It gets more complicated and difficult if you want to detect that a single
series has disappeared. In this case, you need to very specific in telling
Prometheus which series *should* exist. Common ways to do this are
- listing them all out with separate absent(x) clauses and specific positive
matchers
- comparing to a previous time (x offset 15m unless x)
- use some other metric that lets you determine what should be there
- generate recording rules to create such a metric
The fundamental challenge here is to distinguish between "this went missing"
and "this went away because of expected changes".
In general, I prefer splitting "metric indicates there is a problem " and
"metric is missing" into two different alerts with separate names and
descriptions. To the one investigating, the difference matters. Additionally
using absent() often results in different label sets because it cannot know
labels for a time series that is absent. This causes trouble with templating
that you sidestep by using separate alert definitions to begin with.
/MR
On Fri, 9 Dec 2022, 08:31 Sebastian Glock, <sebagloc...@gmail.com
<mailto:sebagloc...@gmail.com> > wrote:
Hi,
I'm having trouble setting up an alert that will send a notification when a
value is different from 0 and the value is missing (i.e. null).
expression:
windows_mscluster_resourcegroup_state {name!~"Available Storage"} != 0 or on()
vector(0)
The alert goes off non-stop. How can I set the metric to send an alert when the
value is different from 0 and is null?
I tried with sum() but not working anyway:
sum(windows_mscluster_resourcegroup_state {name!~"Available Storage"} != 0) or
on() vector(0)
Thanks for replies!
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to prometheus-users+unsubscr...@googlegroups.com
<mailto:prometheus-users+unsubscr...@googlegroups.com> .
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/9fbc7d5d-c7ce-4b93-b653-733cac798956n%40googlegroups.com
<https://groups.google.com/d/msgid/prometheus-users/9fbc7d5d-c7ce-4b93-b653-733cac798956n%40googlegroups.com?utm_medium=email&utm_source=footer>
.
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/04e501d90bab%242c5659b0%2485030d10%24%40gmail.com.
