Hi David, Thanks for the KIP. Kafka is a complicated system and I am constantly learning about it. I understand that rack-aware assignment is costly but I am surprised that it only acts when fetch-from-follower is active.
I would have thought that the sweet spot for this is to assign partitions for which the LEADER is in the client's rack. This is how I had envisaged rack-aware assignment working for share consumers, which of course do not support fetch-from-follower. The idea was to assign partitions to share consumers in the same rack if possible, to avoid cross-rack consumption. If we get around to rack-aware assignment for share groups, that's how it will work. Having a cluster-wide setting for disabling rack-aware assignments for consumer groups and defaulting it to false suggests to me that rack-aware assignment for consumer groups as specified doesn't really work as it should. That may well be the case today :( Thanks, Andrew On 2026/10/05 09:43:50 Lucas Brutschy via dev wrote: > Hi David, > > Thanks for the KIP. This is a nice simplification for the common case > (and share + streams in all cases). > > LB1: The compatibility section covers both broker upgrade and > downgrade. Does the same apply to flipping the config? > > LB2: The config is broker-wide, while the benefit depends on the > topics and consumers of each group (i.e., fetch-from-follower is in > use, and replicas do not cover all racks). Did you consider making it > a group-level configuration? > > LB3: Streams groups get the one-time rebalance on upgrade/downgrade as > well (but not on a config flip). Could the compatibility section point > this out explicitly? > > LB4: The javadoc of SubscribedTopicDescriber#racksForPartition() says > it returns the racks of the partition's replicas, and an empty set > only when the topic does not exist. With the config disabled it would > return an empty set for existing topics, which seems to go against > that contract. It's a minor incompatibility, but maybe worth pointing > out that we are changing this contract. > > Cheers, > Lucas > > On Fri, Oct 2, 2026 at 2:18 PM David Jacot <[email protected]> wrote: > > > > Hi all, > > > > I'd like to discuss a small KIP to make server-side rack-aware > > assignment opt-in. > > > > Rack-aware assignment only helps when consumers fetch from followers > > and the replicas of a partition do not cover every rack, yet the group > > coordinator pays for it for every group today. The KIP adds a broker > > configuration, off by default, to enable it for consumer groups. > > > > https://urldefense.com/v3/__https://cwiki.apache.org/confluence/x/d6fwGg__;!!Ayb5sqE7!oWp60py9PBN1N8wp4x5R7udjo0amdMa274UmguolRNYE4oTtWOfcShqENUTSDgkdPekBat50P3iyGJeqL3E$ > > > > Best, > > David >
