Re: [VOTE] KIP-714: Client metrics and observability

2023-12-09 Thread Andrew Schofield
I’d like to summarise the minor changes we made to KIP-714 as we completed the code in Kafka 3.7.0. * Introduced “*” to differentiate “all metrics subscribed” from “no metrics subscribed”. * Corrected the ACL operation for AlterConfigs to ALTER_CONFIGS on CLUSTER. * Removed the “block” option

Re: [VOTE] KIP-714: Client metrics and observability

2023-10-16 Thread Andrew Schofield
The vote for KIP-714 has now concluded and the KIP is APPROVED. The votes are: Binding: +4 (Jason, Matthias, Sophie, Jun) Non-binding: +3 (Milind, Kirk, Philip) -1 (Ryanne) This KIP aims to improve monitoring and troubleshooting of client performance by enabling clients to push metrics

Re: [VOTE] KIP-714: Client metrics and observability

2023-10-13 Thread Jun Rao
Hi, Andrew, Thanks for the KIP. +1 from me too. Jun On Wed, Oct 11, 2023 at 4:00 PM Sophie Blee-Goldman wrote: > This looks great! +1 (binding) > > Sophie > > On Wed, Oct 11, 2023 at 1:46 PM Matthias J. Sax wrote: > > > +1 (binding) > > > > On 9/13/23 5:48 PM, Jason Gustafson wrote: > > >

Re: [VOTE] KIP-714: Client metrics and observability

2023-10-11 Thread Sophie Blee-Goldman
This looks great! +1 (binding) Sophie On Wed, Oct 11, 2023 at 1:46 PM Matthias J. Sax wrote: > +1 (binding) > > On 9/13/23 5:48 PM, Jason Gustafson wrote: > > Hey Andrew, > > > > +1 on the KIP. For many users of Kafka, it may not be fully understood > how > > much of a challenge client

Re: [VOTE] KIP-714: Client metrics and observability

2023-10-11 Thread Matthias J. Sax
+1 (binding) On 9/13/23 5:48 PM, Jason Gustafson wrote: Hey Andrew, +1 on the KIP. For many users of Kafka, it may not be fully understood how much of a challenge client monitoring is. With tens of clients in a cluster, it is already difficult to coordinate metrics collection. When there are

Re: [VOTE] KIP-714: Client metrics and observability

2023-09-13 Thread Jason Gustafson
Hey Andrew, +1 on the KIP. For many users of Kafka, it may not be fully understood how much of a challenge client monitoring is. With tens of clients in a cluster, it is already difficult to coordinate metrics collection. When there are thousands of clients, and when the cluster operator has no

Re: [VOTE] KIP-714: Client metrics and observability

2023-09-13 Thread Philip Nee
Hey Andrew - Thank you for taking the time to reply to my questions. I'm just adding some notes to this discussion. 1. epoch: It can be helpful to know the delta of the client side and the actual leader epoch. It is helpful to understand why sometimes commit fails/client not making progress. 2.

Re: [VOTE] KIP-714: Client metrics and observability

2023-09-12 Thread Andrew Schofield
Hi Philip, Thanks for your vote and interest in the KIP. KIP-714 does not introduce any new client metrics, and that’s intentional. It does tell how that all of the client metrics can have their names transformed into equivalent "telemetry metric names”, and then potentially used in metrics

Re: [VOTE] KIP-714: Client metrics and observability

2023-09-08 Thread Philip Nee
Hey Andrew - +1 but I don't have a binding vote! It took me a while to go through the KIP. Here are some of my notes during the reading: *Metrics* - Should we care about the client's leader epoch? There is a case where the user recreates the topic, but the consumer thinks it is still the same

Re: [VOTE] KIP-714: Client metrics and observability

2023-09-08 Thread Andrew Schofield
Bumping the voting thread for KIP-714. So far, we have: Non-binding +2 (Milind and Kirk), non-binding -1 (Ryanne) Thanks, Andrew > On 4 Aug 2023, at 09:45, Andrew Schofield wrote: > > Hi, > After almost 2 1/2 years in the making, I would like to call a vote for > KIP-714 >

Re: [VOTE] KIP-714: Client metrics and observability

2023-08-09 Thread Ryanne Dolan
-1, non-binding, for reasons previously stated. Ryanne On Fri, Aug 4, 2023, 3:46 AM Andrew Schofield < andrew_schofield_j...@outlook.com> wrote: > Hi, > After almost 2 1/2 years in the making, I would like to call a vote for > KIP-714 ( >

Re: [VOTE] KIP-714: Client metrics and observability

2023-08-09 Thread Kirk True
Hi Andrew, +1 (non-binding) This is a huge step in enabling end-to-end observability for users and hopefully even help us get a better idea where we can improvement the client behavior. And +100 re: librdkafka team involvement. Thanks! > On Aug 8, 2023, at 4:00 AM, Milind Luthra > wrote:

Re: [VOTE] KIP-714: Client metrics and observability

2023-08-08 Thread Milind Luthra
Hi Andrew, thanks for working on the KIP. +1 (non binding) Thanks, Milind On Fri, Aug 4, 2023 at 2:16 PM Andrew Schofield < andrew_schofield_j...@outlook.com> wrote: > Hi, > After almost 2 1/2 years in the making, I would like to call a vote for > KIP-714 ( >

[VOTE] KIP-714: Client metrics and observability

2023-08-04 Thread Andrew Schofield
Hi, After almost 2 1/2 years in the making, I would like to call a vote for KIP-714 (https://cwiki.apache.org/confluence/display/KAFKA/KIP-714%3A+Client+metrics+and+observability). This KIP aims to improve monitoring and troubleshooting of client performance by enabling clients to push metrics

Re: [VOTE] KIP-714: Client Metrics and Observability

2022-05-24 Thread Jason Gustafson
+1 Thanks Magnus! On Tue, May 17, 2022 at 5:43 AM Magnus Edenhill wrote: > Hey all, > > It's that time of year again where we re-restart this vote thread after > some additional > discussions on the disco thread and minor adjustments to the > KIP. > > We're currently at +5 (non-binding) and -1

Re: [VOTE] KIP-714: Client Metrics and Observability

2022-05-17 Thread Magnus Edenhill
Hey all, It's that time of year again where we re-restart this vote thread after some additional discussions on the disco thread and minor adjustments to the KIP. We're currently at +5 (non-binding) and -1 (non-binding) votes. Please cast your votes, people. Thanks, Magnus Den tors 3 mars

Re: [VOTE] KIP-714: Client Metrics and Observability

2022-03-03 Thread Julien Chanaud
+1 As a member of a team which operates several Kafka clusters, I am unequipped when it comes to troubleshooting issues with project teams that did not understand the importance of configuring client-side monitoring. Kafka represents a fraction of their work and they don't have enough experience,

Re: [VOTE] KIP-714: Client Metrics and Observability

2022-01-26 Thread rifer...@riferrei.com
+1 I think this KIP solves a problem that has been around for some time with Kafka deployments, which is the ability to assess the current state of a Kafka architecture but looking at the whole picture. I also share other folks' concerns regarding adding runtime dependencies to the clients;

Re: [VOTE] KIP-714: Client Metrics and Observability

2022-01-26 Thread Magnus Edenhill
Hi all, it's been a while and there's been some more discussions of the KIP which have been addressed on the KIP page. I think it's a good time to revive this vote thread and get things moving. We're currently at +3 (non-binding) and -1 (non-binding) votes. Regards, Magnus Den mån 1 nov.

RE: Re: [VOTE] KIP-714: Client Metrics and Observability

2021-11-05 Thread Igor Buzatovic
+1 We also have a lot of clients using our central Kafka cluster, and it would be great to have client metrics so we can provide end-to-end monitoring. Igor Buzatović Porsche Digital On 2021/11/01 20:19:20 J Rivers wrote: > +1 > > Thank you for the KIP! > > Our organization runs kafka at large

Re: [VOTE] KIP-714: Client Metrics and Observability

2021-11-01 Thread J Rivers
+1 Thank you for the KIP! Our organization runs kafka at large scale in a multi-tenant configuration. We actually have many other enterprises connecting up to our system to retrieve stream data. These feeds vary greatly in volume and velocity. The peak rates are a multiplicative factor of the

Re: [VOTE] KIP-714: Client Metrics and Observability

2021-10-18 Thread Ryanne Dolan
-1 Ryanne On Mon, Oct 18, 2021, 4:30 AM Magnus Edenhill wrote: > Hi all, > > I'd like to start a vote on KIP-714. > https://cwiki.apache.org/confluence/x/2xRRCg > > Discussion thread: > https://www.mail-archive.com/dev@kafka.apache.org/msg119000.html > > Thanks, > Magnus >

Re: [VOTE] KIP-714: Client Metrics and Observability

2021-10-18 Thread Anna McDonald
Hi MagnUs, Thanks for the KIP. +1 (non-binding) Cheers, Anna On Mon, Oct 18, 2021, 5:30 AM Magnus Edenhill wrote: > Hi all, > > I'd like to start a vote on KIP-714. > https://cwiki.apache.org/confluence/x/2xRRCg > > Discussion thread: >

[VOTE] KIP-714: Client Metrics and Observability

2021-10-18 Thread Magnus Edenhill
Hi all, I'd like to start a vote on KIP-714. https://cwiki.apache.org/confluence/x/2xRRCg Discussion thread: https://www.mail-archive.com/dev@kafka.apache.org/msg119000.html Thanks, Magnus