Viktor Nikitash created KAFKA-15401:
---------------------------------------
Summary: Segment with corrupted index should not be uploaded to
remote storage
Key: KAFKA-15401
URL: https://issues.apache.org/jira/browse/KAFKA-15401
Project: Kafka
Issue Type: Improvement
Components: core
Affects Versions: 3.6.0
Reporter: Viktor Nikitash
Fix For: 3.6.0
During data disk full event, there could be a situation with index corruption.
There are existing functions which perform sanity checks to TimeIndex [1],
TxnIndex [2], OffsetIndex [2]. The idea is performing same checks before in
RemoteLogManager before we upload segment to remote storage [4].
Resources:
[1]
[TimeIndex::sanityCheck()|https://github.com/apache/kafka/blob/88d2c4460a1c8c8cf5dbcc9edb43f42fe898ca00/storage/src/main/java/org/apache/kafka/storage/internals/log/TimeIndex.java#L73]
[2]
[TransationIndex::sanityCheck()|https://github.com/apache/kafka/blob/88d2c4460a1c8c8cf5dbcc9edb43f42fe898ca00/storage/src/main/java/org/apache/kafka/storage/internals/log/TransactionIndex.java#L187]
[3][OffsetIndex::sanityCheck()|[https://github.com/apache/kafka/blob/88d2c4460a1c8c8cf5dbcc9edb43f42fe898ca00/storage/src/main/java/org/apache/kafka/storage/internals/log/OffsetIndex.java#L78]]
[4][RemoteLogManager::copyLogSegmentsToRemote()|https://github.com/apache/kafka/blob/88d2c4460a1c8c8cf5dbcc9edb43f42fe898ca00/core/src/main/java/kafka/log/remote/RemoteLogManager.java#L649]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
