[
https://issues.apache.org/jira/browse/KAFKA-3438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15508184#comment-15508184
]
ASF GitHub Bot commented on KAFKA-3438:
---------------------------------------
GitHub user vahidhashemian opened a pull request:
https://github.com/apache/kafka/pull/1894
KAFKA-3438: Rack-aware replica assignment should warn of overloaded brokers
When using the `--generate` option of the replica reassignment tool the
balance of replicas across brokers is checked (only if rack-aware is enabled)
and in case an imbalance is detected, a proper warning message is appended to
the output of the tool (which is the suggested assignment).
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/vahidhashemian/kafka KAFKA-3438
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/kafka/pull/1894.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1894
----
commit 8e71e7f3aaafa189f95228b6f2bb41d765d1bed0
Author: Vahid Hashemian <vahidhashem...@us.ibm.com>
Date: 2016-09-20T23:21:02Z
KAFKA-3438: Rack-aware replica assignment should warn of overloaded brokers
When using the `--generate` option of the replica reassignment tool the
balance
of replicas across brokers is checked (only if rack-aware is enabled) and
in case
an imbalance is detected, a proper warning message is appended to the
suggested
reassignment.
----
> Rack Aware Replica Reassignment should warn of overloaded brokers
> -----------------------------------------------------------------
>
> Key: KAFKA-3438
> URL: https://issues.apache.org/jira/browse/KAFKA-3438
> Project: Kafka
> Issue Type: Improvement
> Affects Versions: 0.10.0.0
> Reporter: Ben Stopford
> Assignee: Vahid Hashemian
> Fix For: 0.10.1.0
>
>
> We've changed the replica reassignment code to be rack aware.
> One problem that might catch users out would be that they rebalance the
> cluster using kafka-reassign-partitions.sh but their rack configuration means
> that some high proportion of replicas are pushed onto a single, or small
> number of, brokers.
> This should be an easy problem to avoid, by changing the rack assignment
> information, but we should probably warn users if they are going to create
> something that is unbalanced.
> So imagine I have a Kafka cluster of 12 nodes spread over two racks with rack
> awareness enabled. If I add a 13th machine, on a new rack, and run the
> rebalance tool, that new machine will get ~6x as many replicas as the least
> loaded broker.
> Suggest a warning be added to the tool output when --generate is called.
> "The most loaded broker has 2.3x as many replicas as the the least loaded
> broker. This is likely due to an uneven distribution of brokers across racks.
> You're advised to alter the rack config so there are approximately the same
> number of brokers per rack" and displays the individual rack→#brokers and
> broker→#replicas data for the proposed move.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
