[
https://issues.apache.org/jira/browse/KAFKA-15351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kamal Chandraprakash updated KAFKA-15351:
-----------------------------------------
Description:
Case-1:
In the FETCH response, the leader-log-start-offset will be piggy-backed. But,
there can be a scenario:
# Leader deleted the remote log segment and updates it's log-start-offset
# Before the replica-2 update it's log-start-offset via FETCH-request, the
leadership changed to replica-2.
# There are no more eligible segments to delete from remote.
# The log-start-offset will be stale (referring to old log-start-offset but
the data was already removed from remote)
# If the consumer starts to read from the beginning of the topic, it will fail
to read.
Case-2:
The old-leader (follower) can delete the remote log segment in middle of leader
election. We need to update the log-start-offset metadata for this case.
See this comment
[https://github.com/apache/kafka/pull/13561#discussion_r1226538752] for more
details.
was:
In the FETCH response, the leader-log-start-offset will be piggy-backed. But,
there can be a scenario:
# Leader deleted the remote log segment and updates it's log-start-offset
# Before the replica-2 update it's log-start-offset via FETCH-request, the
leadership changed to replica-2.
# There are no more eligible segments to delete from remote.
# The log-start-offset will be stale (referring to old log-start-offset but
the data was already removed from remote)
# If the consumer starts to read from the beginning of the topic, it will fail
to read.
> Update log-start-offset after leader election for topics enabled with remote
> storage
> ------------------------------------------------------------------------------------
>
> Key: KAFKA-15351
> URL: https://issues.apache.org/jira/browse/KAFKA-15351
> Project: Kafka
> Issue Type: Sub-task
> Reporter: Kamal Chandraprakash
> Priority: Major
>
> Case-1:
> In the FETCH response, the leader-log-start-offset will be piggy-backed. But,
> there can be a scenario:
> # Leader deleted the remote log segment and updates it's log-start-offset
> # Before the replica-2 update it's log-start-offset via FETCH-request, the
> leadership changed to replica-2.
> # There are no more eligible segments to delete from remote.
> # The log-start-offset will be stale (referring to old log-start-offset but
> the data was already removed from remote)
> # If the consumer starts to read from the beginning of the topic, it will
> fail to read.
>
> Case-2:
> The old-leader (follower) can delete the remote log segment in middle of
> leader election. We need to update the log-start-offset metadata for this
> case.
> See this comment
> [https://github.com/apache/kafka/pull/13561#discussion_r1226538752] for more
> details.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
