Jason Gustafson created KAFKA-7414:
--------------------------------------
Summary: Do not fail broker on out of range offsets in replica
fetcher
Key: KAFKA-7414
URL: https://issues.apache.org/jira/browse/KAFKA-7414
Project: Kafka
Issue Type: Improvement
Components: replication
Reporter: Jason Gustafson
Assignee: Jason Gustafson
In the replica fetcher, we have logic to detect the case when the follower's
offset is ahead of the leader's. If unclean leader election is not enabled, we
raise a fatal error and kill the broker.
This behavior is inconsistent depending on the message format. With
KIP-101/KIP-279, upon becoming a follower, the replica would use leader epoch
information to reconcile the end of the log with the leader and simply
truncate. Additionally, with the old format, the check is not really
bulletproof for detecting data loss since the unclean leader's end offset might
have already caught up to the follower's offset at the time of its initial
fetch or when it queries for the current log end offset.
To make the logic consistent, we could raise a fatal error whenever the
follower has to truncate below the high watermark. However, the fatal error is
probably overkill and it would be better to log a warning since most of the
damage is already done if the leader has already been elected and this causes a
huge blast radius.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
