If you have pretty balanced traffic on each partition and have set auto.leader.rebalance.enabled to true or false, you might not need to do further workload balance.
However, in most cases you probably still need to do some sort of load balancing based on the traffic and disk utilization of each broker. You might want to do leader migration and/or partition reassignment. Leader migration is a cheaper rebalance and mostly addresses CPU and Network unbalance. Partition reassignment is a much more expensive operation as it moves actual data, this can help with disk utilization in addition to CPU and network. Thanks, Jiangjie (Becket) Qin On 7/15/15, 5:19 PM, "Sandy Waters" <[email protected]> wrote: >Hi all, > >Do I need to load balance against the brokers? I am using the python >driver and it seems to only want a single kafka broker host. However, in >a >situation where I have 10 brokers, is it still fine to just give it one >host. Does zookeeper and kafka handle the load balancing and redirect my >push somewhere else? > >Would it hurt if I load balanced with Nginx and had it do round robin to >the brokers? > >Much thanks for any help. > >-Sandy
