[prometheus-users] Re: Change label of specific metric inside prometheus and VM database

[Brian Candler]

[Talking about prometheus only here]

You can change labels within PromQL expressions in queries:
https://prometheus.io/docs/prometheus/latest/querying/functions/#label_replace

And you can use recording rules to generate new timeseries with the result 
of those queries.

But changing the labels *inside* the database on *already scraped* data? I 
haven't seen anything like that.

If you can export and re-ingest the corrected data, e.g. using backfilling 
,
 
then you can delete the old timeseries 
.

On Friday, 17 March 2023 at 15:24:22 UTC Lukáš Míča wrote:

> Hello,
> I would like to ask you, if is it possible additionaly change label values 
> of metrics which were already scraped inside prometheus and Victoria 
> Metrics database.
> Thank you for the response
> Lukáš
>

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/6ad2c170-ba3a-4032-95e6-a693ef138111n%40googlegroups.com.

[prometheus-users] Re: Query data is empty when step is 1h in query_range api

[Brian Candler]

You should never use any scrape interval greater than 2 minutes.  
Prometheus considers data more than 5 minutes old as "stale", unless you've 
tweaked other settings (and it's not recommended).
https://www.robustperception.io/keep-it-simple-scrape_interval-id/

Don't worry about storage.  Prometheus compresses extremely efficiently, so 
even if you scrape the same data 30 times, the delta between them is zero 
and it will take almost no storage space at all.

On Friday, 17 March 2023 at 15:23:50 UTC Bo Liu wrote:

> I set scrape_interval to 1h, if prometheu is pulled from 00.30.00, then 
> data's timestamp is 00.30.00, 01.30.00, 02.30.00
>
>
> What I am trying to do is use query_range api to query from that day's 
> 00.00.00 to 23.59.59, and step is per hour(because scrape_interval is 1h,I 
> think it's more reasonable)
>
>
> But I get empty result, I think when I run this query, it check data at 
> 00.00.00 but data at 00.30.00, then he misses every data point.
>
>
> I didn't add grafana  because I'm not familiar with it, which make this 
> problem more complex.
>
> Currently I use default TSDB as prometheus database because I didn't 
> decide to use whick database already.
>
>
> So is there anyway to do this in promql or prometheus api?
>
>
> PS: Please don't mind my pool English
>
>
>
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/daf2e0ed-eab1-41ad-aa70-bd78bf8ae502n%40googlegroups.com.

Re: [prometheus-users] Query data is empty when step is 1h in query_range api

[Stuart Clark]

On 17/03/2023 10:33, Bo Liu wrote:


I set scrape_interval to 1h, if prometheu is pulled from 00.30.00, 
then data's timestamp is 00.30.00, 01.30.00, 02.30.00



What I am trying to do is use query_range api to query from that day's 
00.00.00 to 23.59.59, and step is per hour(because scrape_interval is 
1h,I think it's more reasonable)



But I get empty result, I think when I run this query, it check data 
at 00.00.00 but data at 00.30.00, then he misses every data point.



I didn't add grafana  because I'm not familiar with it, which make 
this problem more complex.


Currently I use default TSDB as prometheus database because I didn't 
decide to use whick database already.



So is there anyway to do this in promql or prometheus api?


PS: Please don't mind my pool English


The maximum scrape interval is about 2 minutes due to the way staleness 
handling works, so anything longer than that will likely result in what 
you are seeing.


--
Stuart Clark

--
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/71f0ba3a-efb2-30ef-5f80-609b69c0bb84%40Jahingo.com.

[prometheus-users] Re: Related to blackbox exporter

[Brian Candler]

> When I am using static config then it shows fine but when I am using 
service discovery for status of service or pod some are coming up some are 
down

Then clearly you're not scraping/probing the same endpoints as you did with 
the static config.

The easiest way to check is to go to the Prometheus web interface and look 
at the "Status > Targets" page and see what *actual* targets you're 
probing: first with the static config, and then secondly with the SD config 
(actually you can do both at the same time, but configure them as separate 
jobs so you can distinguish them by the "job" label).  The first column 
shows the URL being scraped and the URL params being passed ("module" and 
"target" are the ones that need to be set for blackbox_exporter, of course).

If you can show your static config, and the service discovery config, the 
problem might be clearer.  My guess is the SD discovery isn't returning 
data in the form that you expect, or that your rewrite rules aren't working 
in the way that you expect.

On Friday, 17 March 2023 at 15:23:28 UTC dikshant rai wrote:

> Hi,
> I tried to configure blackbox exporter for checking api status with 
> prometheus on kubernetes. When I am using static config then it shows fine 
> but when I am using service discovery for status of service or pod some are 
> coming up some are down. I used telnet to check connectivity as well that 
> is fine but still don't know where it is failing.
> Please help.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/02ef4935-44e3-4554-82cd-26da54f30e1bn%40googlegroups.com.

[prometheus-users] Change label of specific metric inside prometheus and VM database

[Lukáš Míča]

Hello,
I would like to ask you, if is it possible additionaly change label values 
of metrics which were already scraped inside prometheus and Victoria 
Metrics database.
Thank you for the response
Lukáš

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/92e1b938-9a9d-4798-b633-444d4751e414n%40googlegroups.com.

[prometheus-users] Query data is empty when step is 1h in query_range api

[Bo Liu]

 

I set scrape_interval to 1h, if prometheu is pulled from 00.30.00, then 
data's timestamp is 00.30.00, 01.30.00, 02.30.00


What I am trying to do is use query_range api to query from that day's 
00.00.00 to 23.59.59, and step is per hour(because scrape_interval is 1h,I 
think it's more reasonable)


But I get empty result, I think when I run this query, it check data at 
00.00.00 but data at 00.30.00, then he misses every data point.


I didn't add grafana  because I'm not familiar with it, which make this 
problem more complex.

Currently I use default TSDB as prometheus database because I didn't decide 
to use whick database already.


So is there anyway to do this in promql or prometheus api?


PS: Please don't mind my pool English





-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/52645cc5-5b8b-4a90-b599-3cd899fb3e60n%40googlegroups.com.

[prometheus-users] Related to blackbox exporter

[dikshant rai]

Hi,
I tried to configure blackbox exporter for checking api status with 
prometheus on kubernetes. When I am using static config then it shows fine 
but when I am using service discovery for status of service or pod some are 
coming up some are down. I used telnet to check connectivity as well that 
is fine but still don't know where it is failing.
Please help.

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/8755e1c0-549c-4b5f-8148-0e29d804bfb5n%40googlegroups.com.