Also if you want to take this further, you can attach an "exemplar" to each bucket - one specific example request which generated a latency point within that range. This makes it very easy to jump to a specific log or trace with a latency within that range.
Google for "prometheus exemplar" for more info. Note that it's currently an experimental feature which needs to be enabled. https://prometheus.io/docs/prometheus/latest/configuration/configuration/#exemplars https://prometheus.io/docs/prometheus/latest/feature_flags/#exemplars-storage On Tuesday, 21 June 2022 at 10:18:27 UTC+1 sup...@gmail.com wrote: > This is the perfect use case for a Histogram. > > Rather than try and sample every N milliseconds, measure every user delay > as a histogram Observe(duration) and have Prometheus collect the histogram > at the usual 15 second interval. > > Also, rather than try and write a script, simply build the Prometheus > client library into your service such that the histogram observer is > integrated. This reduces the overhead to the minimum. > > On Tue, Jun 21, 2022 at 10:33 AM Nemanja Delic <deln...@gmail.com> wrote: > >> Main thing is to detect delays in user activity - since they are >> connecting remotely we are trying to catch response times in their regular >> activity. We are able to get this data - but don't see how this can be >> ingested in Prometheus. >> > >> On Tue, Jun 21, 2022 at 10:24 AM Ben Kochie <sup...@gmail.com> wrote: >> >>> Do you really need every sample every 10ms? Or maybe you need to put >>> your measurements into a histogram for more efficient aggregation? >>> >>> What are you actually measuring and what is your actual alert criteria? >>> >>> On Tue, Jun 21, 2022 at 10:18 AM dc3o <deln...@gmail.com> wrote: >>> >>>> We're having a script that measures a user activity on the server. >>>> Script outputs the data in text file(can make it be prometheus friendly). >>>> Problem is that intervals are really short ~ 10ms. Is prometheus proper >>>> tool for scraping these targets? Goal is to graph this data and make sure >>>> I >>>> can alert on some criteria. >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Prometheus Users" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to prometheus-use...@googlegroups.com. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/prometheus-users/1af3ea74-625e-47f3-9ddd-9dcd9a87758dn%40googlegroups.com >>>> >>>> <https://groups.google.com/d/msgid/prometheus-users/1af3ea74-625e-47f3-9ddd-9dcd9a87758dn%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/d3539815-f7a8-4e2e-9b7e-b2c25f6b22a6n%40googlegroups.com.
