[
https://issues.apache.org/jira/browse/KAFKA-8522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17416231#comment-17416231
]
Jun Rao commented on KAFKA-8522:
--------------------------------
The implementation design can be found at KIP-534:
[https://cwiki.apache.org/confluence/display/KAFKA/KIP-534%3A+Retain+tombstones+and+transaction+markers+for+approximately+delete.retention.ms+milliseconds]
> Tombstones can survive forever
> ------------------------------
>
> Key: KAFKA-8522
> URL: https://issues.apache.org/jira/browse/KAFKA-8522
> Project: Kafka
> Issue Type: Improvement
> Components: log cleaner
> Reporter: Evelyn Bayes
> Assignee: Richard Yu
> Priority: Minor
> Fix For: 3.1.0
>
>
> This is a bit grey zone as to whether it's a "bug" but it is certainly
> unintended behaviour.
>
> Under specific conditions tombstones effectively survive forever:
> * Small amount of throughput;
> * min.cleanable.dirty.ratio near or at 0; and
> * Other parameters at default.
> What happens is all the data continuously gets cycled into the oldest
> segment. Old records get compacted away, but the new records continuously
> update the timestamp of the oldest segment reseting the countdown for
> deleting tombstones.
> So tombstones build up in the oldest segment forever.
>
> While you could "fix" this by reducing the segment size, this can be
> undesirable as a sudden change in throughput could cause a dangerous number
> of segments to be created.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
