[
https://issues.apache.org/jira/browse/KAFKA-13549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17461738#comment-17461738
]
Matthias J. Sax commented on KAFKA-13549:
-----------------------------------------
You could also just do the KIP. Should not be too controversial IMHO. – We just
need one, because adding a new config is a public API change.
> Add "delete interval" config
> ----------------------------
>
> Key: KAFKA-13549
> URL: https://issues.apache.org/jira/browse/KAFKA-13549
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Reporter: Matthias J. Sax
> Priority: Major
> Labels: needs-kip
>
> Kafka Streams uses "delete record" requests to aggressively purge data from
> repartition topics. Those request are sent each time we commit.
> For at-least-once with a default commit interval of 30 seconds, this works
> fine. However, for exactly-once with a default commit interval of 100ms, it's
> very aggressive. The main issue is broker side, because the broker logs every
> "delete record" request, and thus broker logs are spammed if EOS is enabled.
> We should consider to add a new config (eg `delete.record.interval.ms` or
> similar) to have a dedicated config for "delete record" requests, to decouple
> it from the commit interval config and allow to purge data less aggressively,
> even if the commit interval is small to avoid the broker side log spamming.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
