[ 
https://issues.apache.org/jira/browse/KAFKA-16073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Manikumar updated KAFKA-16073:
------------------------------
    Fix Version/s: 3.7.1

> Kafka Tiered Storage: Consumer Fetch Error Due to Delayed localLogStartOffset 
> Update During Segment Deletion
> ------------------------------------------------------------------------------------------------------------
>
>                 Key: KAFKA-16073
>                 URL: https://issues.apache.org/jira/browse/KAFKA-16073
>             Project: Kafka
>          Issue Type: Bug
>          Components: core, Tiered-Storage
>    Affects Versions: 3.6.1
>            Reporter: hzh0425
>            Assignee: hzh0425
>            Priority: Major
>              Labels: KIP-405, kip-405, tiered-storage
>             Fix For: 3.6.2, 3.8.0, 3.7.1
>
>
> The identified bug in Apache Kafka's tiered storage feature involves a 
> delayed update of {{localLogStartOffset}} in the 
> {{UnifiedLog.deleteSegments}} method, impacting consumer fetch operations. 
> When segments are deleted from the log's memory state, the 
> {{localLogStartOffset}} isn't promptly updated. Concurrently, 
> {{ReplicaManager.handleOffsetOutOfRangeError}} checks if a consumer's fetch 
> offset is less than the {{{}localLogStartOffset{}}}. If it's greater, Kafka 
> erroneously sends an {{OffsetOutOfRangeException}} to the consumer.
> In a specific concurrent scenario, imagine sequential offsets: {{{}offset1 < 
> offset2 < offset3{}}}. A client requests data at {{{}offset2{}}}. While a 
> background deletion process removes segments from memory, it hasn't yet 
> updated the {{LocalLogStartOffset}} from {{offset1}} to {{{}offset3{}}}. 
> Consequently, when the fetch offset ({{{}offset2{}}}) is evaluated against 
> the stale {{offset1}} in {{{}ReplicaManager.handleOffsetOutOfRangeError{}}}, 
> it incorrectly triggers an {{{}OffsetOutOfRangeException{}}}. This issue 
> arises from the out-of-sync update of {{{}localLogStartOffset{}}}, leading to 
> incorrect handling of consumer fetch requests and potential data access 
> errors.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Reply via email to