[
https://issues.apache.org/jira/browse/KAFKA-15615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17833359#comment-17833359
]
appchemist commented on KAFKA-15615:
------------------------------------
HI, [~kirktrue]
I want take this issue.
Do you have any potential solutions in mind?
I sent my approach as a PR.
> Improve handling of fetching during metadata updates
> ----------------------------------------------------
>
> Key: KAFKA-15615
> URL: https://issues.apache.org/jira/browse/KAFKA-15615
> Project: Kafka
> Issue Type: Improvement
> Components: clients, consumer
> Reporter: Kirk True
> Priority: Major
> Labels: consumer-threading-refactor, fetcher
> Fix For: 3.8.0
>
>
> [During a review of the new
> fetcher|https://github.com/apache/kafka/pull/14406#discussion_r1333393941],
> [~junrao] found what appears to be an opportunity for optimization.
> When a fetch response receives an error about partition leadership, fencing,
> etc. a metadata refresh is triggered. However, it takes time for that refresh
> to occur, and in the interim, it appears that the consumer will blindly
> attempt to fetch data for the partition again, in kind of a "definition of
> insanity" type of way. Ideally, the consumer would have a way to temporarily
> ignore those partitions, in a way somewhat like the "pausing" approach so
> that they are skipped until the metadata refresh response is fully processed.
> This affects both the existing KafkaConsumer and the new
> PrototypeAsyncConsumer.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
