Re: [prometheus-developers] Re: auto discover of targets in Prometheus

2021-02-21 Thread Gajendra D Ambi
the ipam sd sounds intriguing but i wonder if this is true with prometheus
on k8s since the configmap is presented as a secret to prometheus.

On Sun, Feb 21, 2021, 6:33 PM Stuart Clark  wrote:

> On 20/02/2021 15:08, Gajendra D Ambi wrote:
> > Hi Team,
> > Yes we do have a IPAM available. When we run prometheus node exporter
> > agent it starts showing up metrics at ip/metrics. I was hoping for a
> > way to give a range of ip addresses to prometheus, rather than adding
> > them each as a static target. What would be the best way for us then!
> > with 100s of baremetal servers, Also these baremetals are managed by
> > openstack, I do see openstack auto discovery but will it also discover
> > bare metal servers managed by openstack or just the VM instances?
> > because prometheus wasn't explicit on that part in the documentation.
> > If you could clarify in here or documentation then it would be awesome.
>
> The OpenStack SD can do both -
>
> https://prometheus.io/docs/prometheus/latest/configuration/configuration/#openstack_sd_config
> - It has support for both hypervisor and instance for all Nova servers.
>
> You can use the File SD
> (
> https://prometheus.io/docs/prometheus/latest/configuration/configuration/#file_sd_config)
>
> in conjunction with your IPAM to manage scrape configs too. A common
> method is to run a regular scheduled job to interrogate the IPAM for
> new, changed or removed devices and then produce JSON or YAML files for
> your jobs - depending on the level of information you could produce
> lists for the SNMP exporter (e.g. network equipment), node exporter
> (Unix servers), MySQL exporter (and other databases), IPMI exporter, etc.
>
> --
> Stuart Clark
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-developers/CAHy9ZTh_Dm%2BNGDbB16Xj7ehHrAmyJSdtNybj8RE346k8B9moDQ%40mail.gmail.com.


Re: [prometheus-developers] Re: auto discover of targets in Prometheus

2021-02-20 Thread Gajendra D Ambi
Hi Team,
Yes we do have a IPAM available. When we run prometheus node exporter agent
it starts showing up metrics at ip/metrics. I was hoping for a way to give
a range of ip addresses to prometheus, rather than adding them each as a
static target. What would be the best way for us then! with 100s of
baremetal servers, Also these baremetals are managed by openstack, I do see
openstack auto discovery but will it also discover bare metal servers
managed by openstack or just the VM instances? because prometheus wasn't
explicit on that part in the documentation. If you could clarify in here or
documentation then it would be awesome.

Thanks and Regards,
https://ambig.one/2/
*Gajendra D Ambi *|AWS-CSAA, VCAP6-DCA, 2xVCA, 7xVCP, VCE-CIA, EMC-ISA,
Comptia Server+/A+, MCSE, 2xMCTS vExpert (2017/2018) |



On Sat, Feb 20, 2021 at 12:56 PM Ben Kochie  wrote:

> Prometheus absolutely does do dynamic discovery.
>
> See the scrape_config section of the docs:
> https://prometheus.io/docs/prometheus/latest/configuration/configuration/#scrape_config
>
> There are 16 available XXX_sd_configs options there. If your source of
> truth isn't listed, you can either write your own adapter using
> file_sd_configs as an API. Or if you think your system is popular enough,
> contribute it to the codebase.
>
> There is a growing list of adapters here:
> https://prometheus.io/docs/operating/integrations/#file-service-discovery
>
> On Fri, Feb 19, 2021 at 11:19 PM Gajendra Ambi  wrote:
>
>> Hi,
>> I too just discovered that it does not do dynamic discovery. I have to
>> tell prometheus to see it by using file based discovery in prometheus. This
>> is very unfortunate where we have 100s of nodes and growing and each group
>> of devs who own them get to add them when they want. Currently it seems We
>> have to do it manually. Thanks for the quick info.
>>
>> On Saturday, February 20, 2021 at 3:33:47 AM UTC+5:30 Julien Pivotto
>> wrote:
>>
>>> On 19 Feb 14:00, Gajendra Ambi wrote:
>>> > Everyone gives the same link
>>> >
>>> https://prometheus.io/docs/prometheus/latest/configuration/configuration
>>> when
>>> > asked about auto discovery of physical nodes but the entire page has
>>> only 2
>>> > occurences of the word auto-discovery.
>>> > We have 700+ physical servers and Prometheus is running on k8s
>>> (targets are
>>> > not part of k8s). Can someone please point to a blog or the actual
>>> part of
>>> > the documentation which has auto discovery of physical nodes, not the
>>> same
>>> > link please. I could not find anything there. It says, you can do it
>>> using
>>> > 1 of the auto discovery methods but the link does not have a single
>>> method
>>> > of auto discovery listed there.
>>>
>>> Hello,
>>>
>>> Prometheus does not do *auto* discovery. We do *service* discovery. You
>>> will need a source of truth where your register your nodes.
>>>
>>>
>>> >
>>> > On Monday, September 3, 2018 at 7:44:49 PM UTC+5:30 HARSH AGARWAL
>>> wrote:
>>> >
>>> > > You can use Prometheus-service-discovery(SD) to automatically
>>> discover the
>>> > > targets.
>>> > >
>>> https://prometheus.io/docs/prometheus/latest/configuration/configuration
>>> > >
>>> > > Using relabel-configs
>>> > > <
>>> https://prometheus.io/docs/prometheus/latest/configuration/configuration/#%3Crelabel_config%3E>,
>>>
>>> > > you can specify which targets you want to
>>> keep/discard/change-labels, etc.
>>> > >
>>> > > Prometheus supports SD for many providers. You can also use your
>>> > > own custom-service-discovery mechanisms using file_sd
>>> > > <
>>> https://prometheus.io/docs/prometheus/latest/configuration/configuration/#%3Cfile_sd_config%3E>
>>>
>>> > > .
>>> > >
>>> > >
>>> > > On Monday, September 3, 2018 at 7:36:28 PM UTC+5:30, HJS wrote:
>>> > >>
>>> > >> For a particular job in Prometheus, it seems like the typical
>>> config is
>>> > >> something like this:
>>> > >>
>>> > >> static_configs:
>>> > >> - targets: ['localhost:9090']
>>> > >>
>>> > >> But in the case where I want a dynamic list of hosts((auto discover
>>> of
>>> > >> targets)), this option is available in prometheus a