Prasanna Kumar P.V. created KAFKA-4886:
------------------------------------------
Summary: In a topic with partitions with replication factor of 1,
when a partition is added AFTER a new broker is added, Kafka does not always
assign new partition to the new broker.
Key: KAFKA-4886
URL: https://issues.apache.org/jira/browse/KAFKA-4886
Project: Kafka
Issue Type: Bug
Components: admin
Affects Versions: 0.10.1.0
Environment: A kafka cluster of 3 KVM VM nodes with RHEL 7.2.
Reporter: Prasanna Kumar P.V.
Bring up kafka cluster of two brokers (say ids 0 and 1). Create a topic 'X'
with two partitions and replication factor 1. Now add a third broker (id 2).
Alter the topic X to have one more partition. Issue is, the newly added
partition (i.e partition 2) does not go to the newly added broker if partition
0 has been allotted to broker id 1 and partition 1 is allotted to broker id 0.
However, if partition 0 leader has been set to broker id 0 and partition 1 to
broker id 1 when creating topic 'X', partition 2 would always go to broker id 2
(a uniform distribution). Due to this, we have to always check and follow up
with partition reassignment when a new partition is created to spread
processing load uniformly in cluster. (During createTopic in AdminUtils.scala,
we let fixedStartIndex become -1, in assignReplicasToBrokersRackAware() we have
val startIndex = if (fixedStartIndex >= 0) fixedStartIndex else
rand.nextInt(arrangedBrokerList.size). So startIndex could be 0 or 1.
When startIndex is 1, during alter topic, we end up in
assignReplicasToBrokersRackAware() which then does: val firstReplicaIndex =
(currentPartitionId + startIndex) % arrangedBrokerList.size. the
firstReplicaIndex becomes either 0 or 2. When it becomes 0, we eventually have
the new partition ending up on broker that already is hosting a partition
instead of getting assigned to broker id 2.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
