[ https://issues.apache.org/jira/browse/KAFKA-15351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Satish Duggana updated KAFKA-15351: ----------------------------------- Description: Case-1: In the FETCH response, the leader-log-start-offset will be piggy-backed. But, there can be a scenario: # Leader deleted the remote log segment and updates it's log-start-offset # Before the replica-2 update it's log-start-offset via FETCH-request, the leadership changed to replica-2. # There are no more eligible segments to delete from remote. # The log-start-offset will be stale (referring to old log-start-offset but the data was already removed from remote) # If the consumer starts to read from the beginning of the topic, it will fail to read. Case-2: The old-leader (follower) can delete the remote log segment in middle of leader election. We need to update the log-start-offset metadata for this case. See this comment [https://github.com/apache/kafka/pull/13561#discussion_r1293081560] for more details. was: Case-1: In the FETCH response, the leader-log-start-offset will be piggy-backed. But, there can be a scenario: # Leader deleted the remote log segment and updates it's log-start-offset # Before the replica-2 update it's log-start-offset via FETCH-request, the leadership changed to replica-2. # There are no more eligible segments to delete from remote. # The log-start-offset will be stale (referring to old log-start-offset but the data was already removed from remote) # If the consumer starts to read from the beginning of the topic, it will fail to read. Case-2: The old-leader (follower) can delete the remote log segment in middle of leader election. We need to update the log-start-offset metadata for this case. See this comment [https://github.com/apache/kafka/pull/13561#discussion_r1226538752] for more details. > Update log-start-offset after leader election for topics enabled with remote > storage > ------------------------------------------------------------------------------------ > > Key: KAFKA-15351 > URL: https://issues.apache.org/jira/browse/KAFKA-15351 > Project: Kafka > Issue Type: Sub-task > Reporter: Kamal Chandraprakash > Priority: Major > > Case-1: > In the FETCH response, the leader-log-start-offset will be piggy-backed. But, > there can be a scenario: > # Leader deleted the remote log segment and updates it's log-start-offset > # Before the replica-2 update it's log-start-offset via FETCH-request, the > leadership changed to replica-2. > # There are no more eligible segments to delete from remote. > # The log-start-offset will be stale (referring to old log-start-offset but > the data was already removed from remote) > # If the consumer starts to read from the beginning of the topic, it will > fail to read. > > Case-2: > The old-leader (follower) can delete the remote log segment in middle of > leader election. We need to update the log-start-offset metadata for this > case. > See this comment > [https://github.com/apache/kafka/pull/13561#discussion_r1293081560] for more > details. -- This message was sent by Atlassian Jira (v8.20.10#820010)