[prometheus-users] Re: snmp_exporter related configurations.

[chintaman...@gmail.com]

Error details, as observed in UI.
An error has occurred while serving metrics: error collecting metric 
Desc{fqName: "snmp_error", help: "Error scraping target", constLabels: {}, 
variableLabels: []}: 
error getting target 192.168.1.2: request timeout (after 3 retries)

On Wednesday, February 22, 2023 at 9:37:31 AM UTC+5:30 
chintaman...@gmail.com wrote:

> Hi All,
> Are there any in-depth detailed documents and videos, which describes 
> snmp_exporter configurations?  including some architecture details, like, 
> 1.  snmp_exporter should be installed on prometheus server/VM or need 
> dedicated server/VM ?
> 2. From how many nodes , snmp_exporter can collect metrics ?
> 3. In case of overload, how can multiple snmp_exporter can connect with 
> single (or multiple) prometheus servers.?
> 4. where exactly is auth: for targets? (in snmp.yml, or other file)
> 5. Is it essential to build using generate?
>
> I saw few references, downloaded snmp_exporter (confirmed that prometheus 
> server is getting SNMP response from targets), did initial configurations, 
> added community string in auth: of snmp.yml , added config to 
> prometheus.yml  
> Still unable to reach to targets. SO wan to recheck steps by referring any 
> detailed document for snmp_exporter
>
> Regards,
>  
>

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/2e71909b--4918-bb9a-74d503265ebfn%40googlegroups.com.

[prometheus-users] snmp_exporter related configurations.

[chintaman...@gmail.com]

Hi All,
Are there any in-depth detailed documents and videos, which describes 
snmp_exporter configurations?  including some architecture details, like, 
1.  snmp_exporter should be installed on prometheus server/VM or need 
dedicated server/VM ?
2. From how many nodes , snmp_exporter can collect metrics ?
3. In case of overload, how can multiple snmp_exporter can connect with 
single (or multiple) prometheus servers.?
4. where exactly is auth: for targets? (in snmp.yml, or other file)
5. Is it essential to build using generate?

I saw few references, downloaded snmp_exporter (confirmed that prometheus 
server is getting SNMP response from targets), did initial configurations, 
added community string in auth: of snmp.yml , added config to 
prometheus.yml  
Still unable to reach to targets. SO wan to recheck steps by referring any 
detailed document for snmp_exporter

Regards,
 

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/ebde1135-d0de-40cd-9dc8-50a3fa62f7e4n%40googlegroups.com.

Re: [prometheus-users] How to keep servers up while computer is off. (Blackbox_Exporter)

[Stuart Clark]

On 21/02/2023 21:07, Sean Coh wrote:

Hi Guys,

I came across an issue where if I turn my computer off the services 
will go down a few minutes later and will not scrape my targets.


I know the blackbox host and port is: *127.0.0.1:9115. *Is there a way 
I can keep my targets scrape while my computer is away?


Do you mean you are switching off the machine which is running Blackbox 
Exporter or Prometheus?


--
Stuart Clark

--
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/f0d23a58-1413-add8-067e-c80aed096b6e%40Jahingo.com.

[prometheus-users] kubeprometheus container login details to delete WAL

['Laxman Soma' via Prometheus Users]

Please suggest on how to login into kubeprometheus container and we are 
using google managed prometheus.

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/fef73161-9de8-4afd-81d6-32cfdbf05a5cn%40googlegroups.com.

[prometheus-users] How to keep servers up while computer is off. (Blackbox_Exporter)

[Sean Coh]

Hi Guys,

I came across an issue where if I turn my computer off the services will go 
down a few minutes later and will not scrape my targets.

I know the blackbox host and port is: *127.0.0.1:9115. *Is there a way I 
can keep my targets scrape while my computer is away? 

Relabel_Configs:
- source_labels: [__address__]
  target_label: __param_target
- source_labels: [__param_target]
   target_label: instance
- target_label: __address__
  replacement: 127.0.0.1:9115 

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/c0d69284-2e04-4851-be53-ff84a81d0081n%40googlegroups.com.

Re: [prometheus-users] metric_relabel_configs not dropping metrics

[Jihui Yang]

Thanks, I found the place to make the changes! Basically for each `kind: 
ServiceMonitor` add `metricRelabelings` to drop labels. Appreciate the help!

On Tuesday, February 21, 2023 at 9:51:19 AM UTC-8 Stuart Clark wrote:

> On 21/02/2023 17:29, Jihui Yang wrote:
> > I didn't find a way to adjust those. If I append scrape config jobs to 
> > the end of the config file, they should be able to overwrite existing 
> > job right?
>
> No. Job configurations are self contained. So metrics scraped by a 
> particular job will have any relabelling rules applied for that 
> particular job only. The only way you can set relabelling rules for a 
> job is by editing the job config. For the Prometheus Operator this is 
> done by adjusting the PodMonitor/ServiceMonitor objects.
>
> -- 
> Stuart Clark
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/05ff5635-840b-4d8d-8663-e495579c7664n%40googlegroups.com.

Re: [prometheus-users] server uptime

[Stuart Clark]

On 21/02/2023 09:56, sri L wrote:

Thanks for your reply Julius Volz.
Yes, we are monitoring servers with Node Exporter. We are looking for 
Uptime average based on calendar Month instead of last 30 days.
For example, February Month it should give the uptime average for 
28days i.e., 1st Feb to 28th feb


On Saturday, February 18, 2023 at 2:11:33 AM UTC+5:30 Julius Volz wrote:

Assuming you are monitoring your servers via something like the
Node Exporter, and you want the trailing 30-day upness percentage,
you could use the Node Exporters "up" metric like this:

    avg_over_time(up{job="node"}[30d]) * 100


You would need to adjust the query to be

avg_over_time(up{job="node"}[28d]) * 100

--
Stuart Clark

--
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/4af97a2d-03c6-3cc5-c173-ec5cd01b6d44%40Jahingo.com.

Re: [prometheus-users] metric_relabel_configs not dropping metrics

[Stuart Clark]

On 21/02/2023 17:29, Jihui Yang wrote:
I didn't find a way to adjust those. If I append scrape config jobs to 
the end of the config file, they should be able to overwrite existing 
job right?


No. Job configurations are self contained. So metrics scraped by a 
particular job will have any relabelling rules applied for that 
particular job only. The only way you can set relabelling rules for a 
job is by editing the job config. For the Prometheus Operator this is 
done by adjusting the PodMonitor/ServiceMonitor objects.


--
Stuart Clark

--
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/523da4f7-4772-b333-1c5d-2e78f345e305%40Jahingo.com.

Re: [prometheus-users] metric_relabel_configs not dropping metrics

[Jihui Yang]

I didn't find a way to adjust those. If I append scrape config jobs to the 
end of the config file, they should be able to overwrite existing job right?

On Monday, February 20, 2023 at 3:21:48 PM UTC-8 Stuart Clark wrote:

> On 20/02/2023 23:14, Jihui Yang wrote:
>
> I'm using prometheus-operator. It only allows loading 
> addionalScrapeConfigs 
> 
>  to 
> append to the end of the config file. The other config jobs were added as 
> part of loading prometheus-operator. I'm not sure I can change those.
>
> The other jobs are probably from PodMonitor & ServiceMonitor objects, so 
> you'd need to adjust those.
>
> -- 
> Stuart Clark
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/0ee9a62a-8178-4990-854c-0487aa0d054dn%40googlegroups.com.

Re: [prometheus-users] fading out sample resolution for samples from longer ago possible?

[Ben Kochie]

This is mostly unnecessary in Prometheus because it uses compression in the
TSDB samples. What would take up a lot of space in an RRD file takes up
very little space in Prometheus.

A basic nearline 20TB HDD can easily store 600,000 series for 10 years at
full 15s resolution.

This is possible because the average sample point size in Prometheus is
about 1.5 bytes per sample. So 1.5 bytes * 5760 samples/day * 365 days * 10
years =~ 30MiB.

So for your example, looking up the data for a single metric over a long
period of time is still pretty cheap. What's actually more difficult is
doing all the index loads for this long period of time. But Prometheus uses
mmap to opportunistically access the data on disk.

On Tue, Feb 21, 2023 at 4:29 AM Christoph Anton Mitterer 
wrote:

> Hey.
>
> I wondered whether one can to with Prometheus something similar that is
> possible with systems using RRD (e.g. Ganlia).
>
> Depending on the kind of metrics, like for those from the node exporter,
> one may want a very high sample resolution (and thus short scraping
> interval) for like the last 2 days,... but the further one goes back the
> less interesting those data becomes, at least in that resolution (ever
> looked a how much IO a server had 2 years ago per 15s)?
>
> What one may however want is a rough overview of these metrics for those
> time periods longer ago, e.g. in order to see some trends.
>
>
> For other values, e.g. the total used disk space on a shared filesystem or
> maybe a tape library, one may not need such high resolution for the last 2
> days, but therefore want the data (with low sample resolution, e.g. 1
> sample per day) going back much longer, like the last 10 years.
>
>
> With Ganglia/RRD it one would then simply use multiple RRDs, each for
> different time spans and with different resolutions... and RRD would
> interpolate it's samples accordingly.
>
>
> Can anything like this be done with Prometheus? Or is that completely out
> of scope?
>
>
> I saw that one can set the retention period, but that seems to affect
> everything.
>
> So even if I have e.g. my low resolution tape library total size, which I
> could scrape only every hour or so, ... it wouldn't really help me.
> In order to keep data for that like the last 10 years, I'd need to set the
> retention time to that.
>
> But then the high resolution samples like from the node exporter would
> also be kept that long (with full resolution).
>
>
> Thanks,
> Chris.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Prometheus Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to prometheus-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/prometheus-users/36e3506c-1fba-48e4-b3d9-ead908767cf2n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/CABbyFmqSw%3DQH-0Lx7GuXAUQawKX8omgyKGC6FPG3%2BcC3hW%3DyKA%40mail.gmail.com.

Re: [prometheus-users] fading out sample resolution for samples from longer ago possible?

[Julien Pivotto]

On 21 Feb 07:31, Stuart Clark wrote:
> On 21/02/2023 03:29, Christoph Anton Mitterer wrote:
> > Hey.
> > 
> > I wondered whether one can to with Prometheus something similar that is
> > possible with systems using RRD (e.g. Ganlia).
> > 
> > Depending on the kind of metrics, like for those from the node exporter,
> > one may want a very high sample resolution (and thus short scraping
> > interval) for like the last 2 days,... but the further one goes back the
> > less interesting those data becomes, at least in that resolution (ever
> > looked a how much IO a server had 2 years ago per 15s)?
> > 
> > What one may however want is a rough overview of these metrics for those
> > time periods longer ago, e.g. in order to see some trends.
> > 
> > 
> > For other values, e.g. the total used disk space on a shared filesystem
> > or maybe a tape library, one may not need such high resolution for the
> > last 2 days, but therefore want the data (with low sample resolution,
> > e.g. 1 sample per day) going back much longer, like the last 10 years.
> > 
> > 
> > With Ganglia/RRD it one would then simply use multiple RRDs, each for
> > different time spans and with different resolutions... and RRD would
> > interpolate it's samples accordingly.
> > 
> > 
> > Can anything like this be done with Prometheus? Or is that completely
> > out of scope?
> > 
> > 
> > I saw that one can set the retention period, but that seems to affect
> > everything.
> > 
> > So even if I have e.g. my low resolution tape library total size, which
> > I could scrape only every hour or so, ... it wouldn't really help me.
> > In order to keep data for that like the last 10 years, I'd need to set
> > the retention time to that.
> > 
> > But then the high resolution samples like from the node exporter would
> > also be kept that long (with full resolution).
> > 
> Prometheus itself cannot do downsampling, but other related projects such as
> Cortex & Thanos have such features.


We would love to have this in the future but it would require careful
planning and design document.


-- 
Julien Pivotto
@roidelapluie

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/Y/SgxLFrcQ4fL0mo%40nixos.

Re: [prometheus-users] server uptime

[sri L]

Thanks for your reply Julius Volz. 
Yes, we are monitoring servers with Node Exporter. We are looking for 
Uptime average based on calendar Month instead of last 30 days. 
For example, February Month it should give the uptime average for 28days 
i.e., 1st Feb to 28th feb

On Saturday, February 18, 2023 at 2:11:33 AM UTC+5:30 Julius Volz wrote:

> Assuming you are monitoring your servers via something like the Node 
> Exporter, and you want the trailing 30-day upness percentage, you could use 
> the Node Exporters "up" metric like this:
>
> avg_over_time(up{job="node"}[30d]) * 100
>
> This is assuming that the scrape interval is not changing over those 30d, 
> as otherwise you would be weighting some periods (the ones with a higher 
> scrape frequency) more than others.
>
> On Fri, Feb 17, 2023 at 5:25 AM sri L  wrote:
>
>> Hi all,
>>
>> I am looking for server uptime percentage metrics on a monthly basis
>> Example: If server is down for 60hrs out of 720hrs of a month the uptime 
>> has to show 91.66% in dashboard
>>
>> Please suggest if you have a relevant expression to serve this purpose
>>
>> Thanks
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Prometheus Users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to prometheus-use...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/prometheus-users/e34b891c-a5ea-43c8-b567-83588354f831n%40googlegroups.com
>>  
>> 
>> .
>>
>
>
> -- 
> Julius Volz
> PromLabs - promlabs.com
>

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/9a0eb1cd-fc79-4eb4-a3fe-d088cc09bc81n%40googlegroups.com.