To balance the leaders, you can run the tool in http://kafka.apache.org/documentation.html#basic_ops_leader_balancing
In the upcoming 0.8.2 release, we have fixed the auto leader balancing logic. So leaders will be balanced automatically. Thanks, Jun On Tue, Oct 21, 2014 at 12:19 AM, Shlomi Hazan <shl...@viber.com> wrote: > trying to reproduce failed: after somewhat long minutes I noticed that the > partition leaders regained balance again, and the only issue left is that > the preferred replica was not balanced as it was before taking the broker > down. meaning, that the output of the topic description shows broker 1 (out > of 3) as preferred replica (first in ISR) in 66% of the cases instead of > expected 33%. > > > > On Mon, Oct 20, 2014 at 11:36 PM, Joel Koshy <jjkosh...@gmail.com> wrote: > > > As Neha mentioned, with rep factor 2x, this shouldn't normally cause > > an issue. > > > > Taking the broker down will cause the leader to move to another > > replica; consumers and producers will rediscover the new leader; no > > rebalances should be triggered. > > > > When you bring the broker back up, unless you run a preferred replica > > leader re-election the broker will remain a follower. Again, there > > will be no effect on the producers or consumers (i.e., no rebalances). > > > > If you can reproduce this easily, can you please send exact steps to > > reproduce and send over your consumer logs? > > > > Thanks, > > > > Joel > > > > On Mon, Oct 20, 2014 at 09:13:27PM +0300, Shlomi Hazan wrote: > > > Yes I did. It is set to 2. > > > On Oct 20, 2014 5:38 PM, "Neha Narkhede" <neha.narkh...@gmail.com> > > wrote: > > > > > > > Did you ensure that your replication factor was set higher than 1? If > > so, > > > > things should recover automatically after adding the killed broker > back > > > > into the cluster. > > > > > > > > On Mon, Oct 20, 2014 at 1:32 AM, Shlomi Hazan <shl...@viber.com> > > wrote: > > > > > > > > > Hi, > > > > > > > > > > Running some tests on 0811 and wanted to see what happens when a > > broker > > > > is > > > > > taken down with 'kill'. I bumped into the situation at the subject > > where > > > > > launching the broker back left him a bit out of the game as far as > I > > > > could > > > > > see using stack driver metrics. > > > > > Trying to rebalance with "verify consumer rebalance" return an > error > > "no > > > > > owner for partition" for all partitions of that topic (128 > > partitions). > > > > > moreover, yet aside from the issue at hand, changing the group name > > to a > > > > > non-existent group returned success. > > > > > taking both the consumers and producers down allowed the rebalance > to > > > > > return success... > > > > > > > > > > And the question is: > > > > > How do you restore 100% state after taking down a broker? what is > the > > > > best > > > > > practice? what needs be checked and what needs be done? > > > > > > > > > > Shlomi > > > > > > > > > > > > > >
