[
https://issues.apache.org/jira/browse/KAFKA-15352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kamal Chandraprakash updated KAFKA-15352:
-----------------------------------------
Description:
In Kafka-14888, the remote log segments are deleted which breaches the
retention time/size before updating the log-start-offset. In middle of
deletion, if the consumer starts to read from the beginning of the topic, then
it will fail to read the messages and UNKNOWN_SERVER_ERROR will be thrown back
to the consumer.
To ensure consistency, similar to local log segments where the actual segments
are deleted after {{{}segment.delete.delay.ms{}}}, we should update the
log-start-offset first before deleting the remote log segment.
See the [PR#13561|https://github.com/apache/kafka/pull/13561] and
[comment|https://github.com/apache/kafka/pull/13561#discussion_r1293086543] for
more details.
Case-2:
The old-leader (follower) can delete the remote log segment in the middle of
leader election. We need to update the log-start-offset metadata for this case.
See this commentÂ
[https://github.com/apache/kafka/pull/13561#discussion_r1293081560]
was:
In Kafka-14888, the remote log segments are deleted which breaches the
retention time/size before updating the log-start-offset. In middle of
deletion, if the consumer starts to read from the beginning of the topic, then
it will fail to read the messages and UNKNOWN_SERVER_ERROR will be thrown back
to the consumer.
To ensure consistency, similar to local log segments where the actual segments
are deleted after {{segment.delete.delay.ms}}, we should update the
log-start-offset first before deleting the remote log segment.
See the [PR#13561|https://github.com/apache/kafka/pull/13561] and
[comment|https://github.com/apache/kafka/pull/13561#discussion_r1293086543] for
more details.
> Ensure consistency while deleting the remote log segments
> ---------------------------------------------------------
>
> Key: KAFKA-15352
> URL: https://issues.apache.org/jira/browse/KAFKA-15352
> Project: Kafka
> Issue Type: Sub-task
> Reporter: Kamal Chandraprakash
> Assignee: Kamal Chandraprakash
> Priority: Blocker
> Fix For: 3.6.0
>
>
> In Kafka-14888, the remote log segments are deleted which breaches the
> retention time/size before updating the log-start-offset. In middle of
> deletion, if the consumer starts to read from the beginning of the topic,
> then it will fail to read the messages and UNKNOWN_SERVER_ERROR will be
> thrown back to the consumer.
> To ensure consistency, similar to local log segments where the actual
> segments are deleted after {{{}segment.delete.delay.ms{}}}, we should update
> the log-start-offset first before deleting the remote log segment.
> See the [PR#13561|https://github.com/apache/kafka/pull/13561] and
> [comment|https://github.com/apache/kafka/pull/13561#discussion_r1293086543]
> for more details.
> Case-2:
> The old-leader (follower) can delete the remote log segment in the middle of
> leader election. We need to update the log-start-offset metadata for this
> case.
> See this commentÂ
> [https://github.com/apache/kafka/pull/13561#discussion_r1293081560]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
