[
https://issues.apache.org/jira/browse/KAFKA-3854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vahid Hashemian updated KAFKA-3854:
-----------------------------------
Description:
There are a couple of issues with regex subscription in the new consumer:
# When consecutive calls are made to new consumer's [regex
subscription|https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java#L850],
like below:
{code}
consumer.subscribe(Pattern.compile("..."), listener);
consumer.poll(0);
consumer.subscribe(Pattern.compile("f.."), listener);
consumer.poll(0);
{code}
the second call fails with the following error:
{code}
Exception in thread "main" java.lang.IllegalStateException: Subscription to
topics, partitions and pattern are mutually exclusive
at
org.apache.kafka.clients.consumer.internals.SubscriptionState.subscribe(SubscriptionState.java:175)
at
org.apache.kafka.clients.consumer.KafkaConsumer.subscribe(KafkaConsumer.java:854)
...
{code}
# Even after the code is tweaked to get around the above issue, only the first
call to regex subscription triggers the subscription's rebalance listener. The
reason is the regex {{subscription}} / {{poll}} does not directly call
{{changeSubscription(...)}} in which {{needsPartitionAssignment}} is set and
causes a rebalance. This method is called only during the first regex
{{subscription}} / {{poll}} when coordinator is unknown and [a {{client.poll}}
call|https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractCoordinator.java#L179]
is made which eventually leads to [a {{changeSubscription}}
call|https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java#L161].
The subsequent call does not reach this point because the coordinator is
already known.
was:
# When consecutive calls are made to new consumer's [regex
subscription|https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java#L850],
like below:
{code}
consumer.subscribe(Pattern.compile("..."), listener);
consumer.poll(0);
consumer.subscribe(Pattern.compile("f.."), listener);
consumer.poll(0);
{code}
the second call fails with the following error:
{code}
Exception in thread "main" java.lang.IllegalStateException: Subscription to
topics, partitions and pattern are mutually exclusive
at
org.apache.kafka.clients.consumer.internals.SubscriptionState.subscribe(SubscriptionState.java:175)
at
org.apache.kafka.clients.consumer.KafkaConsumer.subscribe(KafkaConsumer.java:854)
at
ConsumerSubscriptionSemantics.tryRegexSubscriptionSemantics(ConsumerSubscriptionSemantics.java:76)
at
ConsumerSubscriptionSemantics.main(ConsumerSubscriptionSemantics.java:88)
{code}
> Consecutive regex subscription calls fail
> -----------------------------------------
>
> Key: KAFKA-3854
> URL: https://issues.apache.org/jira/browse/KAFKA-3854
> Project: Kafka
> Issue Type: Bug
> Components: consumer
> Reporter: Vahid Hashemian
> Assignee: Vahid Hashemian
>
> There are a couple of issues with regex subscription in the new consumer:
> # When consecutive calls are made to new consumer's [regex
> subscription|https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java#L850],
> like below:
> {code}
> consumer.subscribe(Pattern.compile("..."), listener);
> consumer.poll(0);
> consumer.subscribe(Pattern.compile("f.."), listener);
> consumer.poll(0);
> {code}
> the second call fails with the following error:
> {code}
> Exception in thread "main" java.lang.IllegalStateException: Subscription to
> topics, partitions and pattern are mutually exclusive
> at
> org.apache.kafka.clients.consumer.internals.SubscriptionState.subscribe(SubscriptionState.java:175)
> at
> org.apache.kafka.clients.consumer.KafkaConsumer.subscribe(KafkaConsumer.java:854)
> ...
> {code}
> # Even after the code is tweaked to get around the above issue, only the
> first call to regex subscription triggers the subscription's rebalance
> listener. The reason is the regex {{subscription}} / {{poll}} does not
> directly call {{changeSubscription(...)}} in which
> {{needsPartitionAssignment}} is set and causes a rebalance. This method is
> called only during the first regex {{subscription}} / {{poll}} when
> coordinator is unknown and [a {{client.poll}}
> call|https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractCoordinator.java#L179]
> is made which eventually leads to [a {{changeSubscription}}
> call|https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java#L161].
> The subsequent call does not reach this point because the coordinator is
> already known.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
