horkyada opened a new pull request, #22661:
URL: https://github.com/apache/kafka/pull/22661

   JIRA: https://issues.apache.org/jira/browse/KAFKA-20732
   
   **Problem**
   
   On a freshly elected leader of a tiered partition, `RemoteLogManager` 
size-retention can run while `highestOffsetInRemoteStorage` is still `-1` — it 
is seeded only by the copy and follower tasks, which race the expiration task 
(both are scheduled with `initialDelay=0` on separate thread pools). 
`UnifiedLog.onlyLocalLogSegmentsSize()` then filters `baseOffset >= -1` and 
counts the entire local log, which is also counted in `remoteLogSizeBytes` for 
the same offsets. The apparent size roughly doubles, a false `retention.bytes` 
breach fires, and in-retention data is deleted from the remote tier and — via 
the resulting `logStartOffset` advance, replicated to followers — the local 
tier on all replicas.
   
   The follower path already guards this: `RLMFollowerTask.execute()` seeds the 
offset with the comment *"so that the local log segments are not deleted before 
they are copied to remote storage."* The leader/expiration path, which runs 
size-retention, had no equivalent step.
   
   **Fix**
   
   In `cleanupExpiredRemoteLogSegments`, when `highestOffsetInRemoteStorage() 
== -1`, seed it from remote metadata (`findHighestRemoteOffset`) before 
computing size-retention. It is a no-op once the offset is already seeded.
   
   **Testing**
   
   Added 
`RemoteLogManagerTest.testSizeRetentionDoesNotOverDeleteOnFreshLeaderUntilHighestRemoteOffsetSeeded`
 — fails without the fix (deletes both copied segments), passes with it. Also 
reproduced and fix-validated on a live 3.9.0 cluster (details in the JIRA).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to