hzh0425 created KAFKA-16073:
-------------------------------
Summary: Tiered Storage Bug: Incorrect Handling of Offset Ranges
During Segment Deletion
Key: KAFKA-16073
URL: https://issues.apache.org/jira/browse/KAFKA-16073
Project: Kafka
Issue Type: Bug
Components: core, Tiered-Storage
Affects Versions: 3.6.1
Reporter: hzh0425
Assignee: hzh0425
Fix For: 3.6.1
This bug pertains to Apache Kafka's tiered storage functionality. Specifically,
it involves a timing issue in the {{UnifiedLog.deleteSegments}} method. The
method first deletes segments from memory but delays updating the
{{{}localLogStartOffset{}}}. Meanwhile, in
{{{}ReplicaManager.handleOffsetOutOfRangeError{}}}, if the fetch offset is less
than {{{}localLogStartOffset{}}}, it triggers the read remote process. However,
if it's greater, an {{OffsetOutOfRangeException}} is sent to the client.
Consider a scenario with concurrent operations, where {{{}offset1 < offset2 <
offset3{}}}. A client requests {{offset2}} while a background thread is
deleting segments. The segments are deleted in memory, but
{{LocalLogStartOffset}} is still at {{offset1}} and not yet updated to
{{{}offset3{}}}. In this state, since {{offset2}} is greater than
{{{}offset1{}}}, {{ReplicaManager.handleOffsetOutOfRangeError}} erroneously
returns an {{OffsetOutOfRangeException}} to the client. This happens because
the system has not yet recognized the new starting offset ({{{}offset3{}}}),
leading to incorrect handling of fetch requests.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
