[
https://issues.apache.org/jira/browse/KAFKA-3297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15169074#comment-15169074
]
ASF GitHub Bot commented on KAFKA-3297:
---------------------------------------
GitHub user noslowerdna opened a pull request:
https://github.com/apache/kafka/pull/979
KAFKA-3297: Fair consumer partition assignment strategy (new consumer)
Pull request for https://issues.apache.org/jira/browse/KAFKA-3297
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/noslowerdna/kafka KAFKA-3297
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/kafka/pull/979.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #979
----
commit 34f113453f03bf2e791181fa0db5d41ea985022a
Author: Andrew Olson <[email protected]>
Date: 2016-02-26T14:15:05Z
KAFKA-3297: Fair consumer partition assignment strategy (new consumer)
----
> More optimally balanced partition assignment strategy (new consumer)
> --------------------------------------------------------------------
>
> Key: KAFKA-3297
> URL: https://issues.apache.org/jira/browse/KAFKA-3297
> Project: Kafka
> Issue Type: Improvement
> Reporter: Andrew Olson
> Assignee: Andrew Olson
>
> While the roundrobin partition assignment strategy is an improvement over the
> range strategy, when the consumer topic subscriptions are not identical
> (previously disallowed but will be possible as of KAFKA-2172) it can produce
> heavily skewed assignments. As suggested
> [here|https://issues.apache.org/jira/browse/KAFKA-2172?focusedCommentId=14530767&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14530767]
> it would be nice to have a strategy that attempts to assign an equal number
> of partitions to each consumer in a group, regardless of how similar their
> individual topic subscriptions are. We can accomplish this by tracking the
> number of partitions assigned to each consumer, and having the partition
> assignment loop assign each partition to a consumer interested in that topic
> with the least number of partitions assigned.
> Additionally, we can optimize the distribution fairness by adjusting the
> partition assignment order:
> * Topics with fewer consumers are assigned first.
> * In the event of a tie for least consumers, the topic with more partitions
> is assigned first.
> The general idea behind these two rules is to keep the most flexible
> assignment choices available as long as possible by starting with the most
> constrained partitions/consumers.
> This JIRA addresses the new consumer. For the original high-level consumer,
> see KAFKA-2435.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
