[
https://issues.apache.org/jira/browse/KAFKA-15695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17780216#comment-17780216
]
Nikhil Ramakrishnan commented on KAFKA-15695:
---------------------------------------------
[~ckamal]
I added an integration test (for case 1) to
[#14649|https://github.com/apache/kafka/pull/14649] with the fix. The
integration test does not pass without the change. Please review! Thanks!
> Local log start offset is not updated on the follower after rebuilding remote
> log auxiliary state
> -------------------------------------------------------------------------------------------------
>
> Key: KAFKA-15695
> URL: https://issues.apache.org/jira/browse/KAFKA-15695
> Project: Kafka
> Issue Type: Bug
> Components: replication, Tiered-Storage
> Affects Versions: 3.6.0
> Reporter: Nikhil Ramakrishnan
> Assignee: Nikhil Ramakrishnan
> Priority: Major
> Labels: KIP-405, tiered-storage
> Fix For: 3.7.0
>
>
> In 3.6, the local log start offset is not updated when reconstructing the
> auxiliary state of the remote log on a follower.
> The impact of this bug is significant because, if this follower becomes the
> leader before the local log start offset has had a chance to be updated,
> reads from any offset between [wrong log start offset; actual log start
> offset] will be routed on the local storage, which does not contain the
> corresponding data. Consumer reads will in this case never be satisfied.
>
> Reproduction case 1:
> # Create cluster with 2 brokers, broker 0 and broker 1.
> # Create a topic topicA with RF=1, 1 partition (topicA-0) and 1 batch per
> segment.
> # Produce 3 records to topicA, such that segment 1 and segment 2 with the
> first two records are copied to remote and deleted from local storage.
> # Reassign replica to add broker 1 to the replica set for topicA-0, and
> elect broker 1 as the leader.
> # Try to consume from the beginning of topicA-0.
>
> Reproduction case 2:
> # Create a cluster with 2 brokers, broker 0 and broker 1.
> # Create a topic topicA with RF=2, 1 partition (topicA-0) and 2 batches per
> segment, with broker 0 as the leader.
> # Stop broker 1, and produce 3 records to topicA, such that segment 1 with
> the first two records are copied to remote and deleted from local storage.
> # Start broker 1, let it catch up with broker 0.
> # Stop broker 0 such that broker 1 is elected as the leader, and try to
> consume from the beginning of topicA-0.
> Consumer read will not be satisfied in these cases because the local log
> start offset is not updated on broker 1 when it builds the auxiliary state of
> the remote log segments.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
