[
https://issues.apache.org/jira/browse/KAFKA-10688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17242772#comment-17242772
]
Guozhang Wang commented on KAFKA-10688:
---------------------------------------
The main reason I did not go through the consumer route (i.e. KAFKA-3370) is
that it does not help resolving the 2.b) case.
If the application for whatever reason did not commit before shutting down,
then it means the shutdown was not graceful (otherwise we should always
commit). In this case, either reset to earliest and cause potential data
duplicates, or reset to latest and cause potential data loss, seem not ideal.
And hence I propose to still treat them as fatal cases.
> Handle accidental truncation of repartition topics as exceptional failure
> -------------------------------------------------------------------------
>
> Key: KAFKA-10688
> URL: https://issues.apache.org/jira/browse/KAFKA-10688
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Reporter: Guozhang Wang
> Assignee: Guozhang Wang
> Priority: Major
>
> Today we always handle InvalidOffsetException from the main consumer by the
> resetting policy assuming they are for source topics. But repartition topics
> are also source topics and should never be truncated and hence cause
> InvalidOffsetException.
> We should differentiate these repartition topics from external source topics
> and treat the InvalidOffsetException from repartition topics as fatal and
> close the whole application.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
