Luke Chen created KAFKA-16711: --------------------------------- Summary: highestOffsetInRemoteStorage is not updated after logDir altering within broker Key: KAFKA-16711 URL: https://issues.apache.org/jira/browse/KAFKA-16711 Project: Kafka Issue Type: Sub-task Affects Versions: 3.7.0 Reporter: Luke Chen Assignee: Luke Chen
We use topicIdPartition as the key for each RLM task. It will cause highestOffsetInRemoteStorage in log not updated after logDir alter completion. The reproducing flow is like this: # tp-0 locating in dirA is the leader of the partition # tp-0 is altering logDir to dirB # tp-0 is copying segments to remote storage (note: the log in dirA) # The logDir altering for tp-0 is completed # remoteLogManager#onLeadershipChange is invoked, copiedOffsetOption is reset to Optional.empty() # The copying segments to remote storage in step 3 for tp-0 is completed, updating copiedOffsetOption to new offset, as well as the log#highestOffsetInRemoteStorage. (note: the log in dirA) # In the next run of RLMTask, the log will be the one in target dir (dirB), and the log#highestOffsetInRemoteStorage (dirB) will be the default value (-1), which will block the log cleanup operation. -- This message was sent by Atlassian Jira (v8.20.10#820010)