Hi, Artem, Thanks for the KIP.
Could you explain the partition level ProducerIdCount a bit more? Does that reflect the number of PIDs ever produced to a partition since the broker is started? Do we reduce the count after a PID expires? Thanks, Jun On Wed, Jun 22, 2022 at 1:08 AM David Jacot <dja...@confluent.io.invalid> wrote: > Hi Artem, > > The KIP LGTM. > > Thanks, > David > > On Tue, Jun 21, 2022 at 9:32 PM Artem Livshits > <alivsh...@confluent.io.invalid> wrote: > > > > If there is no other feedback I'm going to start voting in a couple days. > > > > -Artem > > > > On Fri, Jun 17, 2022 at 3:50 PM Artem Livshits <alivsh...@confluent.io> > > wrote: > > > > > Thank you for your feedback. Updated the KIP and added the Rejected > > > Alternatives section. > > > > > > -Artem > > > > > > On Fri, Jun 17, 2022 at 1:16 PM Ismael Juma <ism...@juma.me.uk> wrote: > > > > > >> If we don't track them separately, then it makes sense to keep it as > one > > >> metric. I'd probably name it ProducerIdCount in that case. > > >> > > >> Ismael > > >> > > >> On Fri, Jun 17, 2022 at 12:04 PM Artem Livshits > > >> <alivsh...@confluent.io.invalid> wrote: > > >> > > >> > Do you propose to have 2 metrics instead of one? Right now we don't > > >> track > > >> > if the producer id was transactional or idempotent and for metric > > >> > collection we'd either have to pay the cost of iterating over > producer > > >> ids > > >> > (which could be a lot) or split the producer map into 2 or cache the > > >> > counts, which complicates the code. > > >> > > > >> > From the monitoring perspective, I think one metric should be good, > but > > >> > maybe I'm missing some scenarios. > > >> > > > >> > -Artem > > >> > > > >> > On Fri, Jun 17, 2022 at 12:28 AM Ismael Juma <ism...@juma.me.uk> > wrote: > > >> > > > >> > > I like the suggestion to have IdempotentProducerCount and > > >> > > TransactionalProducerCount metrics. > > >> > > > > >> > > Ismael > > >> > > > > >> > > On Thu, Jun 16, 2022 at 2:27 PM Artem Livshits > > >> > > <alivsh...@confluent.io.invalid> wrote: > > >> > > > > >> > > > Hi Ismael, > > >> > > > > > >> > > > Thank you for your feedback. Yes, this is counting the number > of > > >> > > producer > > >> > > > ids tracked by the partition and broker. Another options I was > > >> > thinking > > >> > > of > > >> > > > are the following: > > >> > > > > > >> > > > - IdempotentProducerCount > > >> > > > - TransactionalProducerCount > > >> > > > - ProducerIdCount > > >> > > > > > >> > > > Let me know if one of these seems better, or I'm open to other > name > > >> > > > suggestions as well. > > >> > > > > > >> > > > -Artem > > >> > > > > > >> > > > On Wed, Jun 15, 2022 at 11:49 PM Ismael Juma <ism...@juma.me.uk > > > > >> > wrote: > > >> > > > > > >> > > > > Thanks for the KIP. > > >> > > > > > > >> > > > > ProducerCount seems like a misleading name since producers > > >> without a > > >> > > > > producer id are not counted. Is this meant to count the > number of > > >> > > > producer > > >> > > > > IDs tracked by the broker? > > >> > > > > > > >> > > > > Ismael > > >> > > > > > > >> > > > > On Wed, Jun 15, 2022, 3:12 PM Artem Livshits < > > >> alivsh...@confluent.io > > >> > > > > .invalid> > > >> > > > > wrote: > > >> > > > > > > >> > > > > > Hello, > > >> > > > > > > > >> > > > > > I'd like to start a discussion on the KIP-847: > > >> > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-847%3A+Add+ProducerCount+metrics > > >> > > > > > . > > >> > > > > > > > >> > > > > > -Artem > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > > >
