AhŠ It seems you are more focusing on producer side workload balanceŠ If that is the case, please ignore my previous comments.
Jiangjie (Becket) Qin On 7/15/15, 6:01 PM, "Jiangjie Qin" <j...@linkedin.com> wrote: >If you have pretty balanced traffic on each partition and have set >auto.leader.rebalance.enabled to true or false, you might not need to do >further workload balance. > >However, in most cases you probably still need to do some sort of load >balancing based on the traffic and disk utilization of each broker. You >might want to do leader migration and/or partition reassignment. > >Leader migration is a cheaper rebalance and mostly addresses CPU and >Network unbalance. Partition reassignment is a much more expensive >operation as it moves actual data, this can help with disk utilization in >addition to CPU and network. > >Thanks, > >Jiangjie (Becket) Qin > >On 7/15/15, 5:19 PM, "Sandy Waters" <sandy.watermell...@gmail.com> wrote: > >>Hi all, >> >>Do I need to load balance against the brokers? I am using the python >>driver and it seems to only want a single kafka broker host. However, in >>a >>situation where I have 10 brokers, is it still fine to just give it one >>host. Does zookeeper and kafka handle the load balancing and redirect my >>push somewhere else? >> >>Would it hurt if I load balanced with Nginx and had it do round robin to >>the brokers? >> >>Much thanks for any help. >> >>-Sandy >
