Re: [prometheus-developers] "dead" metrics

2021-06-06 Thread Stuart Clark

On 04/06/2021 13:09, 'Christian' via Prometheus Developers wrote:

Hi @all,

don't know if this is the right place to ask.

How does Prometheus in general deal with "dead" metrics.
With "dead" I mean the node exporter will deliver the same value over 
and over again,

because the logic behind is broken for some reason.

From a developer perspective this would be easy to deal with by 
putting a timestamp inside the metric to check if we are looking on 
old values and then raise an alert.


I guess that's a complete newbie question and there is something in 
place already ?
I think it really depends on what you mean by "dead metrics" from the 
node exporter. Are you meaning custom metrics you are adding via the 
textfile collector? If so, you also get a timestamp metric for when the 
file last changed, which you could alert against (if not changed within 
a certain time period). If you are meaning something different you could 
alert using the changes function - alert if the changes over the past 
time period were 0.


--
Stuart Clark

--
You received this message because you are subscribed to the Google Groups 
"Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-developers/5347da7e-6097-3242-506c-6998a9b653a7%40Jahingo.com.


[prometheus-developers] "dead" metrics

2021-06-04 Thread 'Christian' via Prometheus Developers
Hi @all,

don't know if this is the right place to ask.

How does Prometheus in general deal with "dead" metrics.
With "dead" I mean the node exporter will deliver the same value over and 
over again,
because the logic behind is broken for some reason.

>From a developer perspective this would be easy to deal with by putting a 
timestamp inside the metric to check if we are looking on old values and 
then raise an alert.

I guess that's a complete newbie question and there is something in place 
already ?

Thanks for your answer.

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-developers/5f905492-cd51-4f61-ab6c-2b31d9d8400bn%40googlegroups.com.