RivenSun created KAFKA-13849:
--------------------------------
Summary: All topics whose cleanup policy contains `compact` are
not executing deleteLogStartOffsetBreachedSegments as expected
Key: KAFKA-13849
URL: https://issues.apache.org/jira/browse/KAFKA-13849
Project: Kafka
Issue Type: Bug
Components: core
Affects Versions: 3.0.1
Reporter: RivenSun
This should be a *regression* bug.
https://issues.apache.org/jira/browse/KAFKA-7400
Currently the logic in LogManager.cleanupLogs() does not follow the above JIRA.
Analyze the source code of LogManager.cleanupLogs():
1. When getting deletableLogs, the logs with `compact` enabled are filtered out
in the two branches of if-else below.
{code:java}
// clean current logs.
val deletableLogs = {
if (cleaner != null) {
// prevent cleaner from working on same partitions when changing cleanup
policy
cleaner.pauseCleaningForNonCompactedPartitions()
} else {
currentLogs.filter {
case (_, log) => !log.config.compact
}
}
}{code}
2. But in the subsequent UnifiedLog.deleteOldSegments method, from the comments
and code logic of this method, the topic of enable `compact` also wants to
execute the deleteLogStartOffsetBreachedSegments method.
{code:java}
/**
* If topic deletion is enabled, delete any local log segments that have either
expired due to time based retention
* or because the log size is > retentionSize.
*
* Whether or not deletion is enabled, delete any local log segments that are
before the log start offset
*/
def deleteOldSegments(): Int = {
if (config.delete) {
deleteLogStartOffsetBreachedSegments() +
deleteRetentionSizeBreachedSegments() +
deleteRetentionMsBreachedSegments()
} else {
deleteLogStartOffsetBreachedSegments()
}
} {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
