Hi Aditya, Thanks for the KIP. I can see the idea behind the KIP but I'm not convinced that it's possible to work out a coherent set of RebalanceCause cases, in particular for the consumer group protocol. In this protocol, rebalances are always "coordinator-initiated". Just because the application calls Consumer.subscribe() resulting in a ConsumerGroupHeartbeat request being sent to the GC, it's not certain that the next ConsumerGroupHeartbeat response will contain the newly assigned partitions. Assignment is asynchronous and the GC makes various optimisations to avoid excessive rebalances for groups with a lot of churn. I'm not sure I could tell the difference between "coordinator-initiated" and "subscription-changed". If you can explain in more detail how each of the cases corresponds to ConsumerGroupHeartbeat response, it would help.
We do need to put more work into the broker-side metrics for rebalance for the consumer group protocol. We'd like more visibility than we have today for sure. Thanks, Andrew On 2026/07/08 21:19:20 Aditya Kousik wrote: > Hi all, > > I’d like to start a discussion on KIP-1364 as a follow-up to KIP-1306. > > KIP link: https://cwiki.apache.org/confluence/x/jYE_Gg > > A Kafka consumer client today cannot tell why a rebalance happened without > grepping logs. The rebalance trigger can help with triaging: > > * max.poll.interval.ms exceeded implies misconfig or delayed poll loop; > * Member fencing points at session timeout > * Coordinator-initiated rebalances are regular occurrences due to join and > leave requests. > * Application calls to Consumer.enforceRebalance()are expected and the > application owns the context. > > Existing rebalance metrics describe frequency and latency but carry no cause > dimension: > > rebalance-rate-per-hour, rebalance-latency-{avg,max}, rebalance-total > last-rebalance-seconds-ago, last-poll-seconds-ago > > So, the cause would be useful in diagnosing issues by surfacing the event to > the user. > > Looking forward to hearing your thoughts on this. > > Best, > Aditya Kousik
