[
https://issues.apache.org/jira/browse/KAFKA-13695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marc updated KAFKA-13695:
-------------------------
Attachment: DeletionConceptTestbed
> Low-traffic topics don't roll (and therefore compact) nor delete tombstones
> ---------------------------------------------------------------------------
>
> Key: KAFKA-13695
> URL: https://issues.apache.org/jira/browse/KAFKA-13695
> Project: Kafka
> Issue Type: Improvement
> Affects Versions: 2.5.0, 3.1.0
> Reporter: Marc
> Priority: Major
>
> I set up a testbed with some partitions and inspected carefully the behaviour
> on the Kafka /data folder.
> It looks like when active segment qualifies for rolling it is not effectively
> being closed until a new record arrives. Thus, it cannot be compacted in a
> timely and deterministic manner by means of max.compaction.lag.ms, for
> instance.
> Again the problem occurs when setting up delete.retention.ms. Once compaction
> happened and the canonical latest state of a key is a unique tombstone on the
> compacted tail, we must wait for an arbitrary record arrival in order for
> deletion to be triggered, just as before.
> I expected log.preallocate property to create a new segment file once we
> marked the current one for rolling and we still got no new records, hoping it
> can be rolled at last, but it seems it has nothing to do.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
