Luke Chen created KAFKA-12464:
---------------------------------
Summary: Enhance constrained sticky Assign algorithm
Key: KAFKA-12464
URL: https://issues.apache.org/jira/browse/KAFKA-12464
Project: Kafka
Issue Type: Improvement
Affects Versions: 2.7.0
Reporter: Luke Chen
Assignee: Luke Chen
In KAFKA-9987, we did a great improvement for the case when all consumers were
subscribed to same set of topics. The algorithm contains 4 phases:
# Reassign as many previously owned partitions as possible, up to the maxQuota
# Fill remaining members up to minQuota
# If we ran out of unassigned partitions before filling all consumers, we need
to start stealing partitions from the over-full consumers at max capacity
# Otherwise we may have run out of unfilled consumers before assigning all
partitions, in which case we should just distribute one partition each to all
consumers at min capacity
Take an example for better understanding:
Current status: 2 consumers (C0, C1), and 10 topic partitions: t1p0, t1p1, ...
t1p9
Suppose, current assignment is:
_C0: t1p0, t1p1, t1p2, t1p3, t1p4_
_C1: t1p5, t1p6, t1p7, t1p8, t1p9_
New, new consumer added: C2, so we'll do:
# Reassign as many previously owned partitions as possible, up to the maxQuota
# Fill remaining members up to minQuota
# If we ran out of unassigned partitions before filling all consumers, we need
to start stealing partitions from the over-full consumers at max capacity
# Otherwise we may have run out of unfilled consumers before assigning all
partitions, in which case we should just distribute one partition each to all
consumers at min capacity
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
