[
https://issues.apache.org/jira/browse/KAFKA-13447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Philip Nee reassigned KAFKA-13447:
----------------------------------
Assignee: Philip Nee
> Consumer should not reuse committed offset after topic recreation
> -----------------------------------------------------------------
>
> Key: KAFKA-13447
> URL: https://issues.apache.org/jira/browse/KAFKA-13447
> Project: Kafka
> Issue Type: Bug
> Reporter: Jason Gustafson
> Assignee: Philip Nee
> Priority: Major
> Labels: consumer, needs-kip
>
> KAFKA-12257 fixes an issue in which the consumer is unable to make progress
> after a topic has been recreated. The problem was that the client could not
> distinguish between stale metadata with a lower leader epoch and a recreated
> topic with a lower leader epoch. With TopicId support in KIP-516, the client
> is able to tell when a topic has been recreated since the new topic will have
> a different ID.
> However, what the patch did not fix is the potential reuse of the current
> offset position on the recreated topic. For example, say that the consumer is
> at offset N when the topic gets recreated. Currently, the consumer will
> continue fetching from offset N after detecting the recreation. The most
> likely result of this is either an offset out of range error or a log
> truncation error, but it is also possible for the offset position to remain
> valid on the recreated topic (say for a low-volume topic where the offsets is
> already low, or a case where the consumer was down for a while).
> To fix this issue completely, we need to store the topicId along with the
> committed offset in __consumer_offsets. This would allow the consumer to
> detect when the offset is no longer relevant for the current topic. We also
> need to decide how to raise this case to the user. If the user has enabled
> automatic offset reset, we can probably use that. Otherwise, we might need a
> new exception type to signal the user that the position needs to be reset.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
