Neha Narkhede created KAFKA-851:
-----------------------------------
Summary: Broken handling of leader and isr request leads to
incorrect high watermark checkpoint file
Key: KAFKA-851
URL: https://issues.apache.org/jira/browse/KAFKA-851
Project: Kafka
Issue Type: Bug
Components: replication
Affects Versions: 0.8
Reporter: Neha Narkhede
Assignee: Neha Narkhede
Priority: Blocker
The broker depends on receiving a list of *all* partitions from the controller
on startup. It uses this information to create a list of partitions that will
get check pointed to the high watermark checkpoint file. However, during a make
follower operation, it adds a partition to the high watermark checkpoint list
only if its leader is alive. Due to this, even if the controller sends a full
list of partitions to the broker, replica manager filters it to keep only those
partitions whose leader is alive. This leads to the high watermark value for
the rest of those partitions to reset to 0. Which, in turn, leads to the
follower to fetch from the beginning of leader's log instead of min(log end
offset, high watermark). The effect of this is very long lag on the replica
fetchers leading to high under replicated partition count
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
