Sarwar Bhuiyan created KAFKA-10760:
--------------------------------------
Summary: In compacted topic with max.compaction.lag.ms, the
segments are not rolled until new messages arrive
Key: KAFKA-10760
URL: https://issues.apache.org/jira/browse/KAFKA-10760
Project: Kafka
Issue Type: Bug
Components: core
Reporter: Sarwar Bhuiyan
Currently, if a compacted topic has min.cleanable.dirty.ratio set to something
low and max.compaction.lag.ms set to a small time, according to KIP
[https://cwiki.apache.org/confluence/display/KAFKA/KIP-354] the expectation is
that the active segment will be rolled regardless or segment.ms or whether new
data has come in to "advance" the time. However, in practice, the current
implementation only rolls the segment when new data which means that there are
situations where the topic is not fully compacted until new data arrives which
may not be until a while later. The implementation can be improved by rolling
the segment just purely based on the max.compaction.lag.ms setting.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
