arnabnandy7 opened a new pull request, #23305: URL: https://github.com/apache/kafka/pull/23305
## Summary KAFKA-19895: This change allows log cleaning to recover without a broker restart after a partition has been marked as uncleanable. When an unexpected exception occurs during cleaning, Kafka marks the affected partition as uncleanable and excludes it from subsequent compaction and deletion passes. This state is retained in memory, so updating the topic’s cleanup or retention configuration does not currently retry the partition. This can cause affected partitions, including `__consumer_offsets` partitions, to grow indefinitely until the broker is restarted. ## Changes - Clear the uncleanable state for a topic’s local partitions when its configuration is updated. - Allow the log cleaner to retry those partitions after the underlying problem has been addressed. - Preserve the uncleanable state of unrelated partitions. - Document the `uncleanable-bytes` metric and recovery procedure. ## Testing Added unit coverage to verify that: - Clearing one uncleanable partition does not affect other quarantined partitions. - Updating a topic configuration clears the uncleanable state. Tests commands: ```shell ./gradlew storage:test --tests org.apache.kafka.storage.internals.log.LogCleanerManagerTest ./gradlew storage:test --tests org.apache.kafka.storage.internals.log.LogManagerTest.testTopicConfigChangeRetriesUncleanablePartitions ./gradlew storage:checkstyleMain storage:checkstyleTest spotlessCheck ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
