[
https://issues.apache.org/jira/browse/KAFKA-18690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17922972#comment-17922972
]
Lianet Magrans commented on KAFKA-18690:
----------------------------------------
Clients (about consumer metadata)
> New consumer doesn't updateLatestMetadata for topics matching new regex
> -----------------------------------------------------------------------
>
> Key: KAFKA-18690
> URL: https://issues.apache.org/jira/browse/KAFKA-18690
> Project: Kafka
> Issue Type: Bug
> Components: clients, consumer
> Reporter: Lianet Magrans
> Assignee: Sean Quah
> Priority: Blocker
> Labels: kip-848-client-support
> Fix For: 4.0.0
>
>
> When the consumer subscribes to the new RE2J regex resolved on the broker,
> it's not keeping the leader and epoch info from the metadata response, so not
> able to properly update the fetch positions.
> After getting the assignment with the topics matching the regex, and
> receiving the topics in a metadata response, the consumer is not retaining
> topics that are assigned with Re2Jpattern so fails with
> Leader for partition {} is unknown for fetching offset {}
>
> This seems to be the reason behind it, not retaining topics assigned with RE2J
> [https://github.com/apache/kafka/blob/d19b60521001697bc9d518cfc3d585e7b78d15a3/clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerMetadata.java#L97
>
> |https://github.com/apache/kafka/blob/d19b60521001697bc9d518cfc3d585e7b78d15a3/clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerMetadata.java#L97]
> So when handling the metadata response that includes the matching topics it
> will never make it to update the leader/epoch info.
> [https://github.com/apache/kafka/blob/d19b60521001697bc9d518cfc3d585e7b78d15a3/clients/src/main/java/org/apache/kafka/clients/Metadata.java#L521]
> To validate with should end with a test similar to
> [https://github.com/apache/kafka/blob/d19b60521001697bc9d518cfc3d585e7b78d15a3/core/src/test/scala/integration/kafka/api/PlaintextConsumerSubscriptionTest.scala#L251]
> but including consumption.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
