Jiangjie Qin created KAFKA-2139:
-----------------------------------
Summary: Add a separate controller messge queue with higher
priority on broker side
Key: KAFKA-2139
URL: https://issues.apache.org/jira/browse/KAFKA-2139
Project: Kafka
Issue Type: Improvement
Reporter: Jiangjie Qin
This ticket is supposed to be working together with KAFKA-2029.
There are two issues with current controller to broker messages.
1. On the controller side the message are sent without synchronization.
2. On broker side the controller messages share the same queue as client
messages.
The problem here is that brokers process the controller messages for the same
partition at different times and the variation could be big. This causes
unnecessary data loss and prolong the preferred leader election / controlled
shutdown/ partition reassignment, etc.
KAFKA-2029 was trying to add a boundary between messages for different
partitions. For example, before leader migration for previous partition
finishes, the leader migration for next partition won't begin.
This ticket is trying to let broker process controller messages faster. So the
idea is have separate queue to hold controller messages, if there are
controller messages, KafkaApi thread will first take care of those messages,
otherwise it will proceed messages from clients.
Those two tickets are not ultimate solution to current controller problems, but
just mitigate them with minor code changes. Moving forward, we still need to
think about rewriting controller in a cleaner way.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
