Jason Gustafson created KAFKA-12636:
---------------------------------------
Summary: Ensure retention still enforced for compacted topics if
cleaning is not enabled
Key: KAFKA-12636
URL: https://issues.apache.org/jira/browse/KAFKA-12636
Project: Kafka
Issue Type: Bug
Reporter: Jason Gustafson
We rely on a periodic task in LogManager to delete old segments of
non-compacted topics which either have breached retention time or which have
been explicitly deleted by a call to DeleteRecords. For compacted topics, we
rely on the cleaning task itself to do the same since a compacted topic may
also be configured with "delete" retention.
If log cleaning is not enabled, we still need to enforce retention semantics
for compacted topics, but the current logic in LogManager excludes them from
consideration:
{code}
// clean current logs.
val deletableLogs = {
if (cleaner != null) {
// prevent cleaner from working on same partitions when changing
cleanup policy
cleaner.pauseCleaningForNonCompactedPartitions()
} else {
currentLogs.filter {
case (_, log) => !log.config.compact
}
}
}
{code}
It seems to me that we should remove the filtering when log cleaning is not
enabled. The logic in `deleteOldSegments` will ensure that only the appropriate
retention checks are made based on the topic configuration.
Of course it's kind of weird for a user to have a compacted topic when the
cleaner is not enabled in the first place.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
