Hi Team, Could someone help me with how to distribute kafka topic replicas evenly across brokers to avoid data skew (disk utilisation).
Regards, Abhishek Singla On Sun, Mar 24, 2024 at 2:36 AM Abhishek Singla <abhisheksingla...@gmail.com> wrote: > Hi Team, > > Kafka version: 2_2.12-2.6.0 > Zookeeper version: 3.8.x > > We have a Kafka Cluster of 12 brokers spread equally across 3 racks. Topic > gets auto created with default num.partitions=6 and replication_factor=3. > It is observed that replicas are equally distributed over racks but within > the rack the replicas are randomly distributed like sometimes 3,3,0,0 or > sometimes 3:2:1 or sometime 2,2,1,1 > > Is there a configuration to evenly distribute replicas across brokers > within a rack, maybe some sort of round robin strategy 2,2,1,1? > > And also it is observed that over time 1 broker ends up having way more > replicas across topics than the other broker in the same rack. Is there a > config for even distribution of replicas across topics also? > > Regards, > Abhishek Singla >
