Hi,
Sorry for the late reply let me go on-by-one, please bear with me. :)
*>I recommend taking that question to an OpenMetrics list.*
Thanks, I will open a thread there too based on the discussion here.
*>Prometheus could independently decide to go beyond what OpenMetrics says*
Since Micrometer uses the Prometheus Java Client this would solve the issue
for us but if it makes sense it would be great to have it standardized
later(?) (in OM).
*>Maybe your point is that exemplars are tied to the _bucket metrics and
not the _count metric?*
Yes, that's exactly what I'm saying with the caveat that I think this would
be useful for Summaries too not only Histograms.
*>How exactly would you change this while remaining backwards-compatible
for existing users?*
As far as I can see, adding Exemplars to _count should be a
backward-compatible change (it is an addition).
Would users be broken because of this?
*>Perhaps the upcoming "native histograms" or "sparse histograms" feature
will suit what you need?*
I'm not sure but I'm not only talking about histograms, I'm also talking
about Summary.
*>In an OM Histogram, the +Inf bucket fulfills exactly the same function*
*as the _count (spec says: "The +Inf bucket counts all requests.") Soif you
would like an examplar on the _count of a Histogram, you can aswell use an
exemplar on the +Inf bucket.*
I think I disagree with the second sentence. Let's say you have an
application where processing the first request is significantly slower than
the rest (lazy init, populating caches, GC, establishing connections,
etc.). In this environment (I think this is true for lots of apps nowadays)
it can easily happen that the +Inf bucket will be populated with an
Exemplar for the first request and it will never get updated because the
app will never be as slow as it was for the first request. Also, nothing
guarantees that the +Inf bucker will have an exemplar, maybe the processing
was faster than that. As far as I understand, exemplars are not like
cumulative "le" counters so incrementing a bucket does not mean updating an
exemplar (quite the opposite, maybe all of the buckets will be incremented
but only one will get a new Exemplar). This is also true for apps that can
get significantly faster over time (i.e.: JIT). I think a solution here
would be give_me_the_last_updated_bucket(my_histogram) or adding Exemplar
to _count.
*>Side note: In Java this would be particularly useful because the popular
Spring Boot framework exposes a Summary http_server_requests_seconds by
default that look like this (no quantiles, just _count and _sum)*
This was actually one of the drivers of this request, users are asking for
this from us. I also find it extremely useful: without this I need to
create an additional counter which does not seem right since I already have
one.
Thanks,
Jonatan
On Tuesday, October 18, 2022 at 6:29:27 AM UTC-7 fab...@fstab.de wrote:
> Side note: In Java this would be particularly useful because the popular
> Spring Boot framework exposes a Summary http_server_requests_seconds by
> default that look like this (no quantiles, just _count and _sum):
>
> # HELP http_server_requests_seconds
> # TYPE http_server_requests_seconds summary
> http_server_requests_seconds_count{exception="None",method="GET",outcome="SUCCESS",status="200",uri="/",}
> 1.0
> http_server_requests_seconds_sum{exception="None",method="GET",outcome="SUCCESS",status="200",uri="/",}
> 1.014687278
>
> I think this is pretty useful, you can get request rates and error rates
> out of it. If Prometheus / OpenMetrics had support for Exemplars on the
> _count, users could find example traces per HTTP status and URI.
>
> Fabian
>
> On Tue, Oct 18, 2022 at 3:05 PM Bjoern Rabenstein <bjo...@rabenste.in>
> wrote:
>
>> On 06.10.22 14:45, 'Fabian Stäber' via Prometheus Developers wrote:
>> >
>> > Great question from the CNCF Slack: What's the reason why we don't
>> allow
>> > Exemplars for _count in Summary metrics?
>> >
>> > What do you think? Any reason why Exemplars don't work in _count in
>> > Summaries? Would that be something we could consider supporting?
>>
>> The _count of a Summary _and_ the _count of a Histogram (both
>> conventional as well as the new native ones) is essentially a counter
>> within the larger "structured" metric of a Summary/Histogram.
>>
>> From that perspective, it should have the option of attaching an
>> examplar, as a regular Counter has, too.
>>
>> My speculation why it doesn't in OpenMetrics:
>>
>> In an OM Histogram, the +Inf bucket fulfills exactly the same function
>> as the _count (spec says: "The +Inf bucket counts all requests.") So
>> if you would like an examplar on the _count of a Histogram, you can as
>> well use an exemplar on the +Inf bucket.
>>
>> That obviously doesn't help in the case of a Summary, but I guess the
>> rationale is that Histograms are generally to be preferred over
>> Summaries, and therefore didn't get the thourough treatment when it
>> came to exemplars.
>>
>>
>> However, even if you really dislike the precalculated quantiles in
>> Summaries, there is still the case of a Summary without quantiles. I
>> think adding exemplars to such a Summary is as much needed as adding
>> exemplars to any regular Counter.
>>
>> --
>> Björn Rabenstein
>> [PGP-ID] 0x851C3DA17D748D03
>> [email] bjo...@rabenste.in
>>
>
--
You received this message because you are subscribed to the Google Groups
"Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to prometheus-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-developers/895413bd-416f-40b5-9268-26d3e394ca6fn%40googlegroups.com.
