Sorry, unclear phrasing – I mean as the one scraping (whether by hand or as a Prometheus server), how would I know what the semantics of the value in the provided exemplar are, whether they are a delta or an absolute value? Again, I think "it's always an absolute value" makes this least surprising.
This overlaps with the discussion we had about exemplars from recording rules – if there was the option to have multiple exemplars for one gauge, it could separately present exemplars for increment, decrement, and set, whichever ones have been used … /MR On Wed, Dec 7, 2022 at 9:50 AM 'Fabian Stäber' via Prometheus Developers < prometheus-developers@googlegroups.com> wrote: > > At scrape time, how would I know which method was used? > > The library keeps a list of Exemplars that are provided at scrape time (in > case of a Gauge the list only has 1 element, because Gauges only have 1 > Exemplar). > > The question is what to put in the list when inc(), dec(), or set() are > called. > > My feeling is also that the total value of the gauge is least surprising. > But that would also mean that for cases like "number of Bytes in a queue" > you'd not see the number of Bytes added as an Exemplar value, just the > total queue size. > > Fabian > > On Wed, Dec 7, 2022 at 10:44 AM Matthias Rampke <matth...@prometheus.io> > wrote: > >> At scrape time, how would I know which method was used? "always the total >> value of the gauge" seems like the least surprising choice to me. If users >> need to track separate exemplars for increment and decrement, they could >> use two counters (basically, a non-native UpDownCounter), which >> conceptually and physically preserves that the final value is the result of >> repeated additions and removals. >> >> /MR >> >> On Wed, Dec 7, 2022 at 7:58 AM 'Fabian Stäber' via Prometheus Developers < >> prometheus-developers@googlegroups.com> wrote: >> >>> Thanks Julien. I guess there are examples for both, having Exemplars >>> with the delta as well as the absolute value. >>> >>> Example 1) Number of Bytes in a queue. You would call >>> myGauge.inc(nBytes) if a message is added to the queue, and >>> myGauge.dec(nBytes) if a message is fetched from the queue. If adding / >>> fetching is done via REST API, it would make sense to have Exemplars for >>> the add/fetch operations. >>> >>> Example 2) An IoT device reaching out to a Web hook for reporting >>> temperature. To track the current temperature, you'd call >>> myGauge.set(temperature). If the IoT device is capable of adding trace >>> headers when calling the Web hook it would be good to have Exemplars of >>> these calls. >>> >>> The Java library supports automatic Exemplars, i.e. users don't call >>> explicit API to add Exemplars, but the library figures out if a current >>> trace context exists and samples Exemplars automatically under the hood. >>> For that, we need a default behavior. So what would a good default be? >>> Always the total value of the Gauge, or always the delta, or make it depend >>> on whether inc(), dec(), or set() was called? >>> >>> Fabian >>> >>> >>> On Tue, Dec 6, 2022 at 11:18 PM Julien Pivotto < >>> roidelapl...@prometheus.io> wrote: >>> >>>> On 06 Dec 23:06, 'Fabian Stäber' via Prometheus Developers wrote: >>>> > Hi, >>>> > >>>> > I'm experimenting with Exemplars for Gauge metrics in client_java >>>> > (background: at the dev summit on 10 November 2022 >>>> > < >>>> https://docs.google.com/document/d/11LC3wJcVk00l8w5P3oLQ-m3Y37iom6INAMEu2ZAGIIE/edit >>>> > >>>> > we decided that "Prometheus will ingest Exemplars on all time >>>> series"). >>>> > >>>> > For comparison: A counter MUST have the following methods >>>> > <https://prometheus.io/docs/instrumenting/writing_clientlibs/#counter >>>> >: >>>> > >>>> > - inc(): Increment the counter by 1 >>>> > - inc(double v): Increment the counter by the given amount. MUST >>>> check >>>> > that v >= 0. >>>> > >>>> > Exemplars will contain the increment, i.e. if inc() is called the >>>> Exemplar >>>> > will have the value 1.0, if inc(v) is called the Exemplar will have >>>> the >>>> > value v. >>>> > >>>> > Now, A gauge MUST have the following methods >>>> > <https://prometheus.io/docs/instrumenting/writing_clientlibs/#gauge>: >>>> > >>>> > - inc(): Increment the gauge by 1 >>>> > - inc(double v): Increment the gauge by the given amount >>>> > - dec(): Decrement the gauge by 1 >>>> > - dec(double v): Decrement the gauge by the given amount >>>> > - set(double v): Set the gauge to the given value >>>> > >>>> > Which value should we choose for Gauge Exemplars? >>>> >>>> That is a good question and my guess would be that it will depend on the >>>> use case. Do you have examples of gauges would would attach exemplars >>>> to? >>>> >>>> -- >>>> Julien Pivotto >>>> @roidelapluie >>>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Prometheus Developers" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to prometheus-developers+unsubscr...@googlegroups.com. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/prometheus-developers/CAPX310ipsufix2s4_P4PXOvzGr4MFDbHaaBmmJZXpnW-k%2Bas6g%40mail.gmail.com >>> <https://groups.google.com/d/msgid/prometheus-developers/CAPX310ipsufix2s4_P4PXOvzGr4MFDbHaaBmmJZXpnW-k%2Bas6g%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- > You received this message because you are subscribed to the Google Groups > "Prometheus Developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to prometheus-developers+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-developers/CAPX310ggv%2BthbzFYx5j71B%2ByyOOy9d-64nPAa00-yNd6jcZbkQ%40mail.gmail.com > <https://groups.google.com/d/msgid/prometheus-developers/CAPX310ggv%2BthbzFYx5j71B%2ByyOOy9d-64nPAa00-yNd6jcZbkQ%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "Prometheus Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-developers+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/CAMV%3D_gaT6_kDoK0TY6QvnMYZ5hrGiSAF_ee2eBgOBA5UuOQ1Ow%40mail.gmail.com.
