[
https://issues.apache.org/jira/browse/KAFKA-12261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17287615#comment-17287615
]
Haruki Okada commented on KAFKA-12261:
--------------------------------------
I see, thank you. Seems I don't have permission to set the assignee of the
ticket, but anyway I've submitted a PR. Please take a look when you have time.
> Splitting partition causes message loss for consumers with
> auto.offset.reset=latest
> -----------------------------------------------------------------------------------
>
> Key: KAFKA-12261
> URL: https://issues.apache.org/jira/browse/KAFKA-12261
> Project: Kafka
> Issue Type: Improvement
> Affects Versions: 2.7.0
> Reporter: Haruki Okada
> Priority: Minor
>
> As of now, auto.offset.reset of ConsumerConfig is "latest" by default.
>
> This could be a pitfall that causes message delivery loss when we split
> topic's partitions like below:
> Say we have a topic-X which have only 1 partition.
> # split topic-X to 2 partitions by kafka-topics.sh --alter --topic topic-X
> --partitions 2 (topic-X-1 is added)
> # producer knows that new partitions are added by refreshing metadata.
> starts to produce to topic-X-1
> # bit later, consumer knows that new partitions are added and triggering
> consumer rebalance, then starts consuming topic-X-1
> *
> ** upon starting consumption, it resets its offset to log-end-offset
> If the producer sent several records before 3, they could be not-delivered to
> the consumer.
>
>
> This behavior isn't preferable in most cases, so it should be documented in
> AUTO_OFFSET_RESET_DOC at least.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
