[
https://issues.apache.org/jira/browse/KAFKA-7190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16582931#comment-16582931
]
Matthias J. Sax commented on KAFKA-7190:
----------------------------------------
I want to through out another thought:
Given the current implementation and API (ignoring in-flight transactions for a
moment), a truncated topic may results in lost PIDs and thus in WARNING logs
client side – however, because the producer know how to handle this situation I
am wondering if logging this at WARN level is the right decision and if DEBUG
logging might be more appropriate. This would address the surface issue of the
ticket. WDYT?
For truncation itself, we could say truncating in-flight transactions is a user
error or not allowed and it's the users fault it the producer crashes (not
sure, if this might be an issue with regard to retention? – I guess not in
practice). Delaying a delete-record-request for an in-flight transaction might
be a good other solution.
What I am getting at is, that caching of PIDs in memory might actually not be
required? Thoughts?
> Under low traffic conditions purging repartition topics cause WARN statements
> about UNKNOWN_PRODUCER_ID
> ---------------------------------------------------------------------------------------------------------
>
> Key: KAFKA-7190
> URL: https://issues.apache.org/jira/browse/KAFKA-7190
> Project: Kafka
> Issue Type: Improvement
> Components: core, streams
> Affects Versions: 1.1.0, 1.1.1
> Reporter: Bill Bejeck
> Assignee: lambdaliu
> Priority: Major
>
> When a streams application has little traffic, then it is possible that
> consumer purging would delete
> even the last message sent by a producer (i.e., all the messages sent by
> this producer have been consumed and committed), and as a result, the broker
> would delete that producer's ID. The next time when this producer tries to
> send, it will get this UNKNOWN_PRODUCER_ID error code, but in this case,
> this error is retriable: the producer would just get a new producer id and
> retries, and then this time it will succeed.
>
> Possible fixes could be on the broker side, i.e., delaying the deletion of
> the produderIDs for a more extended period or on the streams side developing
> a more conservative approach to deleting offsets from repartition topics
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
