Thanks a lot Jonatan.
Just a quick heads-up: We have a Prometheus dev summit on Thursday next
week, and I put this on the agenda:
https://docs.google.com/document/d/11LC3wJcVk00l8w5P3oLQ-m3Y37iom6INAMEu2ZAGIIE/edit
Fabian
On Fri, Nov 4, 2022 at 6:46 AM Jonatan Ivanov <jonatan.iva...@gmail.com>
wrote:
> Hi,
>
> Sorry for the late reply let me go on-by-one, please bear with me. :)
>
> *>I recommend taking that question to an OpenMetrics list.*
> Thanks, I will open a thread there too based on the discussion here.
>
> *>Prometheus could independently decide to go beyond what OpenMetrics says*
> Since Micrometer uses the Prometheus Java Client this would solve the
> issue for us but if it makes sense it would be great to have it
> standardized later(?) (in OM).
>
> *>Maybe your point is that exemplars are tied to the _bucket metrics and
> not the _count metric?*
> Yes, that's exactly what I'm saying with the caveat that I think this
> would be useful for Summaries too not only Histograms.
>
> *>How exactly would you change this while remaining backwards-compatible
> for existing users?*
> As far as I can see, adding Exemplars to _count should be a
> backward-compatible change (it is an addition).
> Would users be broken because of this?
>
> *>Perhaps the upcoming "native histograms" or "sparse histograms" feature
> will suit what you need?*
> I'm not sure but I'm not only talking about histograms, I'm also talking
> about Summary.
>
> *>In an OM Histogram, the +Inf bucket fulfills exactly the same function*
>
>
> *as the _count (spec says: "The +Inf bucket counts all requests.") Soif
> you would like an examplar on the _count of a Histogram, you can aswell use
> an exemplar on the +Inf bucket.*
>
> I think I disagree with the second sentence. Let's say you have an
> application where processing the first request is significantly slower than
> the rest (lazy init, populating caches, GC, establishing connections,
> etc.). In this environment (I think this is true for lots of apps nowadays)
> it can easily happen that the +Inf bucket will be populated with an
> Exemplar for the first request and it will never get updated because the
> app will never be as slow as it was for the first request. Also, nothing
> guarantees that the +Inf bucker will have an exemplar, maybe the processing
> was faster than that. As far as I understand, exemplars are not like
> cumulative "le" counters so incrementing a bucket does not mean updating an
> exemplar (quite the opposite, maybe all of the buckets will be incremented
> but only one will get a new Exemplar). This is also true for apps that can
> get significantly faster over time (i.e.: JIT). I think a solution here
> would be give_me_the_last_updated_bucket(my_histogram) or adding Exemplar
> to _count.
>
> *>Side note: In Java this would be particularly useful because the popular
> Spring Boot framework exposes a Summary http_server_requests_seconds by
> default that look like this (no quantiles, just _count and _sum)*
> This was actually one of the drivers of this request, users are asking for
> this from us. I also find it extremely useful: without this I need to
> create an additional counter which does not seem right since I already have
> one.
>
> Thanks,
> Jonatan
>
> On Tuesday, October 18, 2022 at 6:29:27 AM UTC-7 fab...@fstab.de wrote:
>
>> Side note: In Java this would be particularly useful because the popular
>> Spring Boot framework exposes a Summary http_server_requests_seconds by
>> default that look like this (no quantiles, just _count and _sum):
>>
>> # HELP http_server_requests_seconds
>> # TYPE http_server_requests_seconds summary
>> http_server_requests_seconds_count{exception="None",method="GET",outcome="SUCCESS",status="200",uri="/",}
>> 1.0
>> http_server_requests_seconds_sum{exception="None",method="GET",outcome="SUCCESS",status="200",uri="/",}
>> 1.014687278
>>
>> I think this is pretty useful, you can get request rates and error rates
>> out of it. If Prometheus / OpenMetrics had support for Exemplars on the
>> _count, users could find example traces per HTTP status and URI.
>>
>> Fabian
>>
>> On Tue, Oct 18, 2022 at 3:05 PM Bjoern Rabenstein <bjo...@rabenste.in>
>> wrote:
>>
>>> On 06.10.22 14:45, 'Fabian Stäber' via Prometheus Developers wrote:
>>> >
>>> > Great question from the CNCF Slack: What's the reason why we don't
>>> allow
>>> > Exemplars for _count in Summary metrics?
>>> >
>>> > What do you think? Any reason why Exemplars don't work in _count in
>>> > Summaries? Would that be something we could consider supporting?
>>>
>>> The _count of a Summary _and_ the _count of a Histogram (both
>>> conventional as well as the new native ones) is essentially a counter
>>> within the larger "structured" metric of a Summary/Histogram.
>>>
>>> From that perspective, it should have the option of attaching an
>>> examplar, as a regular Counter has, too.
>>>
>>> My speculation why it doesn't in OpenMetrics:
>>>
>>> In an OM Histogram, the +Inf bucket fulfills exactly the same function
>>> as the _count (spec says: "The +Inf bucket counts all requests.") So
>>> if you would like an examplar on the _count of a Histogram, you can as
>>> well use an exemplar on the +Inf bucket.
>>>
>>> That obviously doesn't help in the case of a Summary, but I guess the
>>> rationale is that Histograms are generally to be preferred over
>>> Summaries, and therefore didn't get the thourough treatment when it
>>> came to exemplars.
>>>
>>>
>>> However, even if you really dislike the precalculated quantiles in
>>> Summaries, there is still the case of a Summary without quantiles. I
>>> think adding exemplars to such a Summary is as much needed as adding
>>> exemplars to any regular Counter.
>>>
>>> --
>>> Björn Rabenstein
>>> [PGP-ID] 0x851C3DA17D748D03
>>> [email] bjo...@rabenste.in
>>>
>> --
> You received this message because you are subscribed to the Google Groups
> "Prometheus Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to prometheus-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/prometheus-developers/895413bd-416f-40b5-9268-26d3e394ca6fn%40googlegroups.com
> <https://groups.google.com/d/msgid/prometheus-developers/895413bd-416f-40b5-9268-26d3e394ca6fn%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
>
--
You received this message because you are subscribed to the Google Groups
"Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to prometheus-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-developers/CAPX310gaMq6HvsPwkgBH0msDcp5eJDavKfUbWnEqoMxUbBehSQ%40mail.gmail.com.
