Hi Ryanne, this proposal stems from a need to improve troubleshooting Kafka issues.
As it currently stands, when an application team is experiencing Kafka service degradation, or the Kafka operator is seeing misbehaving clients, there are plenty of steps that needs to be taken before any client-side metrics can be observed at all, if at all: - Is the application even collecting client metrics? If not it needs to be reconfigured or implemented, and restarted; a restart may have business impact, and may also temporarily? remedy the problem without giving any further insight into what was wrong. - Are the desired metrics collected? Where are they stored? For how long? Is there enough correlating information to map it to cluster-side metrics and events? Does the application on-call know how to find the collected metrics? - Export and send these metrics to whoever knows how to interpret them. In what format? Are all relevant metadata fields provided? The KIP aims to solve all these obstacles by giving the Kafka operator the tools to collect this information. Regards, Magnus Den tis 15 juni 2021 kl 02:37 skrev Ryanne Dolan <ryannedo...@gmail.com>: > Magnus, I think such a substantial change requires more motivation than is > currently provided. As I read it, the motivation boils down to this: you > want your clients to phone-home unless they opt-out. As stated in the KIP, > "there are plenty of existing solutions [...] to send metrics [...] to a > collector", so the opt-out appears to be the only motivation. Am I missing > something? > > Ryanne > > On Wed, Jun 2, 2021 at 7:46 AM Magnus Edenhill <mag...@edenhill.se> wrote: > > > Hey all, > > > > I'm proposing KIP-714 to add remote Client metrics and observability. > > This functionality will allow centralized monitoring and troubleshooting > of > > clients and their internals. > > > > Please see > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-714%3A+Client+metrics+and+observability > > > > Looking forward to your feedback! > > > > Regards, > > Magnus > > >
