[
https://issues.apache.org/jira/browse/KAFKA-15538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17807323#comment-17807323
]
Lianet Magrans edited comment on KAFKA-15538 at 1/16/24 7:45 PM:
-----------------------------------------------------------------
Hey [~phuctran], this ticket is not fully implemented yet. There are some bits
of it at the consumer level already, that [~kirktrue] worked on just as an
initial approach, but that needs to be reviewed and make sure nothing else need
to be wired up to the HeartbeatRequestManager, to ensure that the list of
topics matching the regex are sent to the broker (see
[here|https://github.com/apache/kafka/blob/dd0916ef9a6276d191196f79176bcb725e1ff9e6/clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java#L537]).
We should also ensure that we re-enable all the pattern subscription related
integration tests, that are currently disabled in the PlainTextAsyncConsumer
(ex.
[testPatternSubscription|https://github.com/apache/kafka/blob/dd0916ef9a6276d191196f79176bcb725e1ff9e6/core/src/test/scala/integration/kafka/api/PlaintextConsumerTest.scala#L356C7-L356C30]).
I will re-open the issue.
Regarding documentation, for the legacy consumer all we have is the java doc
(afaik). For the new consumer, we do have a one-pager on the AK wiki
[here|https://cwiki.apache.org/confluence/display/KAFKA/Consumer+threading+refactor+design].
Hope it helps!
was (Author: JIRAUSER300183):
Hey [~phuctran], this ticket is not fully implemented yet. There are some bits
of it at the consumer level already, that [~kirktrue] worked on just as an
initial approach, but that needs to be reviewed and wired up to the
HeartbeatRequestManager, to ensure that the list of topics matching the regex
are sent to the broker (see
[here|https://github.com/apache/kafka/blob/dd0916ef9a6276d191196f79176bcb725e1ff9e6/clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java#L537]).
We should also ensure that we re-enable all the pattern subscription related
integration tests, that are currently disabled in the PlainTextAsyncConsumer
(ex.
[testPatternSubscription|https://github.com/apache/kafka/blob/dd0916ef9a6276d191196f79176bcb725e1ff9e6/core/src/test/scala/integration/kafka/api/PlaintextConsumerTest.scala#L356C7-L356C30]).
I will re-open the issue.
Regarding documentation, for the legacy consumer all we have is the java doc
(afaik). For the new consumer, we do have a one-pager on the AK wiki
[here|https://cwiki.apache.org/confluence/display/KAFKA/Consumer+threading+refactor+design].
Hope it helps!
> Client support for java regex based subscription
> ------------------------------------------------
>
> Key: KAFKA-15538
> URL: https://issues.apache.org/jira/browse/KAFKA-15538
> Project: Kafka
> Issue Type: Sub-task
> Components: clients, consumer
> Reporter: Lianet Magrans
> Assignee: Phuc Hong Tran
> Priority: Major
> Labels: kip-848, kip-848-client-support
> Fix For: 3.8.0
>
>
> When using subscribe with a java regex (Pattern), we need to resolve it on
> the client side to send the broker a list of topic names to subscribe to.
> Context:
> The new consumer group protocol uses [Google
> RE2/J|https://github.com/google/re2j] for regular expressions and introduces
> new methods in the consumer API to subscribe using a `SubscribePattern`. The
> subscribe using a java `Pattern` will be still supported for a while but
> eventually removed.
> * When the subscribe with SubscriptionPattern is used, the client should
> just send the regex to the broker and it will be resolved on the server side.
> * In the case of the subscribe with Pattern, the regex should be resolved on
> the client side.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
