Try removing the proxy, within a cluster Prometheus should be able to connect to the pods directly. Consider using Kubernetes SD to discover the daemonset pods dynamically.
/MR On Sat, Mar 6, 2021, 10:02 suyog kulkarni <suyogkulkarni...@gmail.com> wrote: > The current node-exporter is setup as daemon-set in kubernetes cluster > same where prometheus is deployed. > > On Saturday, March 6, 2021 at 2:30:20 PM UTC+5:30 suyog kulkarni wrote: > >> Current prometheus version is 2.20. There is proxy settings also. >> Here are my prometheus-config. >> >> - job_name: spc-targets-node-exporter >> proxy_url: <Proxy-URL> >> scrape_interval: 15s >> scrape_timeout: 30s >> static_configs: >> - targets: >> - pce-9100:9100 >> - pce-9101:9100 >> - pce-9102:9100 >> - pce-9103:9100 >> - pce-9104:9100 >> . >> . >> . >> - pce-N:9100 >> >> >> On Saturday, March 6, 2021 at 1:59:11 PM UTC+5:30 matt...@prometheus.io >> wrote: >> >>> How long does it take when you request the metrics directly from node >>> exporter ? Can you exclude one collector at a time to see which one is the >>> slow one? >>> >>> Generally, node exporter doesn't "do" much other than collect metrics >>> from the kernel; if it is slow most likely something else is too. >>> >>> "context timeout exceeded" is Go's words for "this took longer than the >>> configured timeout". The only way to make it go away is to make whatever is >>> happening take less time than the timeout, or extend the timeout to be more >>> than it takes. >>> >>> /MR >>> >>> On Sat, Mar 6, 2021, 09:14 Julien Pivotto <roidel...@prometheus.io> >>> wrote: >>> >>>> Can we know more about your setup? Which version of prometheus do you >>>> use? Do you use TLS? Do you directly target the node exporters or is there >>>> a proxy in the middle? >>>> >>>> Thanks >>>> >>>> Le sam. 6 mars 2021 à 07:54, suyog kulkarni <suyogkul...@gmail.com> a >>>> écrit : >>>> >>>>> Hello , >>>>> >>>>> I have prometheus deployed as container and currently i am trying to >>>>> scrape 100 static/dynamic endpoints in single job and the corresponding >>>>> scrape interval is 20s and timeout is 15s. with this interval and timeout >>>>> all my endpoints are showing context-deadline-exceeded. While if i change >>>>> the scrape timeout and scrape interval to 50s each then my 100 endpoints >>>>> are up. my target endpoint is node-expoter with heavy list of metrics. >>>>> I dont want to change the scrape interval or scrape timeout. How do i >>>>> get rid of context-deadline-exceeded without changing the scrape interval >>>>> and scrape timeout. >>>>> Thanks in advance. >>>>> >>>>> Regards, >>>>> Suyog Kulkarni >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Prometheus Users" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to prometheus-use...@googlegroups.com. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/prometheus-users/374154c1-c27a-4ee1-ae24-a41922034db7n%40googlegroups.com >>>>> <https://groups.google.com/d/msgid/prometheus-users/374154c1-c27a-4ee1-ae24-a41922034db7n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Prometheus Users" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to prometheus-use...@googlegroups.com. >>>> >>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/prometheus-users/CAFJ6V0r_T5%2BuF%3DYf6Sh%2BF6ZqTT9khx5yXDCczrU9_FoOQsRdow%40mail.gmail.com >>>> <https://groups.google.com/d/msgid/prometheus-users/CAFJ6V0r_T5%2BuF%3DYf6Sh%2BF6ZqTT9khx5yXDCczrU9_FoOQsRdow%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> -- > You received this message because you are subscribed to the Google Groups > "Prometheus Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to prometheus-users+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-users/2e7978e9-d702-4820-9ddd-c6568b69c304n%40googlegroups.com > <https://groups.google.com/d/msgid/prometheus-users/2e7978e9-d702-4820-9ddd-c6568b69c304n%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/CAMV%3D_gZSbJ1crTQW%3DV%2B3omWF4ERkSjWZ2-%2BwLZ%2BLvWKQ8X9BEQ%40mail.gmail.com.
