Re: [prometheus-developers] Multiple metrics path for Prometheus

[Julius Volz]

(I mean /a, /b, etc. of course, not a, b, ...)

On Sat, Jan 30, 2021 at 10:49 PM Julius Volz 
wrote:

> On Fri, Jan 29, 2021 at 3:50 PM Vitaly Shupak 
> wrote:
>
>> This is possible to do using relabel_configs:
>>
>> scrape_configs:
>>   - job_name: 'job'
>> static_configs:
>> - targets:
>> - 'example.com/a'
>> - 'example.com/b'
>> - 'example.com/c'
>> - 'example.com/d'
>> relabel_configs:
>> - source_labels: [__address__]
>>   target_label: __metrics_path__
>>   regex: '(.*)/(.*)'
>>   replacement: '/$2'
>> - source_labels: [__address__]
>>   target_label: instance
>> - source_labels: [__address__]
>>   regex: '(.*)/(.*)'
>>   replacement: '$1'
>>   target_label: __address__
>>
>
> This does not require relabeling, as you can just attach a
> "__metrics_path__" label directly to each target, e.g.:
>
> scrape_configs:
>   - job_name: 'job'
> static_configs:
> - targets:
> - 'example.com'
>   labels:
> __metrics_path__: a
> - targets:
> - 'example.com'
>   labels:
> __metrics_path__: b
> - targets:
> - 'example.com'
>   labels:
> __metrics_path__: c
> - targets:
> - 'example.com'
>   labels:
> __metrics_path__: b
>
> I'd probably still just group them under different scrape configs though,
> as you will get different metrics from each of the endpoints, and thus
> you'll likely want to group them under a different job name (although you
> could also override the "job" label like above, in one scrape config).
>
>
>>
>> On Monday, December 14, 2020 at 1:32:23 PM UTC-5 bjo...@rabenste.in
>> wrote:
>>
>>> There is probably same nuance in arguing if and when this is a good
>>> idea and when not.
>>>
>>> But in fact, the famous
>>> https://github.com/kubernetes/kube-state-metrics is doing it. It's not
>>> using different paths, but different ports, but that's kind of
>>> similar.
>>>
>>> On the Prometheus side, however, you need separate scrape
>>> targets. There is currently no way of "iterating" through multiple
>>> ports or paths of a target. From the Prometheus side, a different
>>> port, a different path, or a different host is just all the same thing
>>> in defining a different target. (And that probably won't change
>>> anytime soon.)
>>>
>>> --
>>> Björn Rabenstein
>>> [PGP-ID] 0x851C3DA17D748D03
>>> [email] bjo...@rabenste.in
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Prometheus Developers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to prometheus-developers+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/prometheus-developers/c604b921-828a-427b-8d10-471ad7ccdae9n%40googlegroups.com
>> 
>> .
>>
>
>
> --
> Julius Volz
> PromLabs - promlabs.com
>


-- 
Julius Volz
PromLabs - promlabs.com

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-developers/CAObpH5yH-UUq9%2B%2Bs4WcDic_QtmU0qYmeq1pwKvgkJY-_oEP89A%40mail.gmail.com.

Re: [prometheus-developers] Multiple metrics path for Prometheus

[Julius Volz]

On Fri, Jan 29, 2021 at 3:50 PM Vitaly Shupak 
wrote:

> This is possible to do using relabel_configs:
>
> scrape_configs:
>   - job_name: 'job'
> static_configs:
> - targets:
> - 'example.com/a'
> - 'example.com/b'
> - 'example.com/c'
> - 'example.com/d'
> relabel_configs:
> - source_labels: [__address__]
>   target_label: __metrics_path__
>   regex: '(.*)/(.*)'
>   replacement: '/$2'
> - source_labels: [__address__]
>   target_label: instance
> - source_labels: [__address__]
>   regex: '(.*)/(.*)'
>   replacement: '$1'
>   target_label: __address__
>

This does not require relabeling, as you can just attach a
"__metrics_path__" label directly to each target, e.g.:

scrape_configs:
  - job_name: 'job'
static_configs:
- targets:
- 'example.com'
  labels:
__metrics_path__: a
- targets:
- 'example.com'
  labels:
__metrics_path__: b
- targets:
- 'example.com'
  labels:
__metrics_path__: c
- targets:
- 'example.com'
  labels:
__metrics_path__: b

I'd probably still just group them under different scrape configs though,
as you will get different metrics from each of the endpoints, and thus
you'll likely want to group them under a different job name (although you
could also override the "job" label like above, in one scrape config).


>
> On Monday, December 14, 2020 at 1:32:23 PM UTC-5 bjo...@rabenste.in wrote:
>
>> There is probably same nuance in arguing if and when this is a good
>> idea and when not.
>>
>> But in fact, the famous
>> https://github.com/kubernetes/kube-state-metrics is doing it. It's not
>> using different paths, but different ports, but that's kind of
>> similar.
>>
>> On the Prometheus side, however, you need separate scrape
>> targets. There is currently no way of "iterating" through multiple
>> ports or paths of a target. From the Prometheus side, a different
>> port, a different path, or a different host is just all the same thing
>> in defining a different target. (And that probably won't change
>> anytime soon.)
>>
>> --
>> Björn Rabenstein
>> [PGP-ID] 0x851C3DA17D748D03
>> [email] bjo...@rabenste.in
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Prometheus Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to prometheus-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/prometheus-developers/c604b921-828a-427b-8d10-471ad7ccdae9n%40googlegroups.com
> 
> .
>


-- 
Julius Volz
PromLabs - promlabs.com

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-developers/CAObpH5wq5gMv3Ao2k0U89sm9TnvkNv1QyDY9zd4qAb9yVgbv5A%40mail.gmail.com.

Re: [prometheus-developers] Multiple metrics path for Prometheus

[Vitaly Shupak]

This is possible to do using relabel_configs:

scrape_configs:
  - job_name: 'job'
static_configs:
- targets:
- 'example.com/a'
- 'example.com/b'
- 'example.com/c'
- 'example.com/d'
relabel_configs:
- source_labels: [__address__]
  target_label: __metrics_path__
  regex: '(.*)/(.*)'
  replacement: '/$2'
- source_labels: [__address__]
  target_label: instance
- source_labels: [__address__]
  regex: '(.*)/(.*)'
  replacement: '$1'
  target_label: __address__


On Monday, December 14, 2020 at 1:32:23 PM UTC-5 bjo...@rabenste.in wrote:

> There is probably same nuance in arguing if and when this is a good 
> idea and when not. 
>
> But in fact, the famous 
> https://github.com/kubernetes/kube-state-metrics is doing it. It's not 
> using different paths, but different ports, but that's kind of 
> similar. 
>
> On the Prometheus side, however, you need separate scrape 
> targets. There is currently no way of "iterating" through multiple 
> ports or paths of a target. From the Prometheus side, a different 
> port, a different path, or a different host is just all the same thing 
> in defining a different target. (And that probably won't change 
> anytime soon.) 
>
> -- 
> Björn Rabenstein 
> [PGP-ID] 0x851C3DA17D748D03 
> [email] bjo...@rabenste.in 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-developers/c604b921-828a-427b-8d10-471ad7ccdae9n%40googlegroups.com.

Re: [prometheus-developers] Multiple metrics path for Prometheus

[Bjoern Rabenstein]

There is probably same nuance in arguing if and when this is a good
idea and when not.

But in fact, the famous
https://github.com/kubernetes/kube-state-metrics is doing it. It's not
using different paths, but different ports, but that's kind of
similar.

On the Prometheus side, however, you need separate scrape
targets. There is currently no way of "iterating" through multiple
ports or paths of a target. From the Prometheus side, a different
port, a different path, or a different host is just all the same thing
in defining a different target. (And that probably won't change
anytime soon.)

-- 
Björn Rabenstein
[PGP-ID] 0x851C3DA17D748D03
[email] bjo...@rabenste.in

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-developers/20201214183220.prbv7alrfzainisr%40jahnn.

Re: [prometheus-developers] Multiple metrics path for Prometheus

[Christian Hoffmann]

Hi,

On 12/9/20 12:42 PM, rakshit gupta wrote:
How can (is it possible) I use multiple endpoints for metrics scraping 
in a job ?


# The HTTP resource path on which to fetch metrics from targets.
  [ metrics_path:  
 
| default = /metrics ]


In my use case, a service( registered to consul) can expose metrics in 
any one of the endpoints - /metrics, /actuator/info, etc, and I need 
generic configuration in Prometheus to scrape data by probably iterating 
through the endpoints. I cannot hard code the endpoint there that 
/metrics is my endpoint.


I don't think this is possible or a good idea. You would need to decide 
which metrics path to use. If you have this information (e.g. during 
service discovery), then you can place those targets in different scrape 
jobs or even re-use the sample scrape job and relabel __metrics_path__ 
accordingly.


If SD doesn't provide this information, another option would be to 
provide it via your configuration management system (e.g. probe your 
targets regularly and see what endpoint is the right one; then record 
this information).


If this information is not available to you, then the only other option 
would be to create scrape jobs for all variants. Two of those three 
would always fail. I don' think it would be a good idea or simple to 
work with.


Kind regards,
Christian

--
You received this message because you are subscribed to the Google Groups 
"Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-developers/e51af840-f9cf-e4c8-a511-e6eac7e4ec4e%40hoffmann-christian.info.

[prometheus-developers] Multiple metrics path for Prometheus

[rakshit gupta]

Hi, 

How can (is it possible) I use multiple endpoints for metrics scraping in a 
job ?

# The HTTP resource path on which to fetch metrics from targets.
 [ metrics_path:  

 
| default = /metrics ]  

In my use case, a service( registered to consul) can expose metrics in any 
one of the endpoints - /metrics, /actuator/info, etc, and I need generic 
configuration in Prometheus to scrape data by probably iterating through 
the endpoints. I cannot hard code the endpoint there that /metrics is my 
endpoint.

Thanks for your help in advance!


-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-developers/cfc2735e-26e5-4930-b20e-3a1915b3d8aan%40googlegroups.com.