Perhaps the most import question to ask is: *why* do you only want to 
scrape once per day?

If you're worrying about storage space consumption: don't.  Prometheus uses 
differential encoding.  If you record the same metric repeatedly with the 
same value, i.e. the difference between the adjacent values is zero, then 
the amount of storage space used is miniscule.  Even if it's not the same 
value, it's extremely highly compressed.  And disk space is cheap.

If you only want to *check* a value every 1 day for alerting, then it's 
perfectly fine to have an alerting rule group with a 1 day evaluation 
interval - even if the underlying metric is collected more frequently. The 
alerting rule will use the most recent value at evaluation time.

If you're worried that the exporter itself is expensive to run on each 
scrape, then you should do one of the other things that Stuart suggested 
(i.e. using textfile collector or push gateway).  On the target node use a 
cron job to update the metric at whatever interval makes sense - but let 
prometheus scrape it as often as it likes.  But you should still scrape the 
cached value every 1-2 minutes.

If it's blackbox_exporter that you're talking about, those scrapes are 
cheap.  So you may as well scrape every 2 minutes.

If you're worried about transient errors causing spurious alarms, then 
adjust your alerting rules (e.g. in the simplest case, add "for: 30m" which 
means that you'll only get a notification if the alert has been active for 
30m *continuously*).

On Friday, 12 August 2022 at 08:31:09 UTC+1 Stuart Clark wrote:

> On 12/08/2022 08:29, nina guo wrote:
> > So if with push gateway or textfile collector, we need to also to 
> > customize our metrics, am I right?
>
> What do you mean?
>
> -- 
> Stuart Clark
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to [email protected].
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/635f5d4d-ff92-45b0-b7ef-291930df5b3cn%40googlegroups.com.

Reply via email to