[jira] [Updated] (KAFKA-3854) Subsequent regex subscription calls fail
[
https://issues.apache.org/jira/browse/KAFKA-3854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ewen Cheslack-Postava updated KAFKA-3854:
-
Fix Version/s: 0.10.0.1
> Subsequent regex subscription calls fail
>
>
> Key: KAFKA-3854
> URL: https://issues.apache.org/jira/browse/KAFKA-3854
> Project: Kafka
> Issue Type: Bug
> Components: consumer
>Reporter: Vahid Hashemian
>Assignee: Vahid Hashemian
> Fix For: 0.10.1.0, 0.10.0.1
>
>
> There are a couple of issues with regex subscription in the new consumer:
> h6. Subsequent Subscriptions Fail
> When consecutive calls are made to new consumer's [regex
> subscription|https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java#L850],
> like below:
> {code}
> consumer.subscribe(Pattern.compile("..."), listener);
> consumer.poll(0);
> consumer.subscribe(Pattern.compile("f.."), listener);
> consumer.poll(0);
> {code}
> the second call fails with the following error:
> {code}
> Exception in thread "main" java.lang.IllegalStateException: Subscription to
> topics, partitions and pattern are mutually exclusive
> at
> org.apache.kafka.clients.consumer.internals.SubscriptionState.subscribe(SubscriptionState.java:175)
> at
> org.apache.kafka.clients.consumer.KafkaConsumer.subscribe(KafkaConsumer.java:854)
> ...
> {code}
> h6. Subsequent Subscriptions Fail to Trigger a Subscription Change and
> Rebalance
> Even after the code is tweaked to get around the above issue, only the first
> call to regex subscription triggers an actual subscription and the
> subscription's rebalance listener. The reason is the regex {{subscription}} /
> {{poll}} does not directly call {{changeSubscription(...)}} in which
> {{needsPartitionAssignment}} is set and causes a rebalance. This method is
> called only during the first regex {{subscription}} / {{poll}} when
> coordinator is unknown and [a {{client.poll}}
> call|https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractCoordinator.java#L179]
> is made which eventually leads to [a {{changeSubscription}}
> call|https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java#L161].
> The subsequent call does not reach this point because the coordinator is
> already known.
> It seems due to the same reason, if a consumer is subscribed to a pattern,
> and later on a new topic is created that matches that pattern, the consumer
> does not become subscribed it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (KAFKA-3854) Subsequent regex subscription calls fail
[
https://issues.apache.org/jira/browse/KAFKA-3854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ewen Cheslack-Postava updated KAFKA-3854:
-
Resolution: Fixed
Fix Version/s: 0.10.1.0
Status: Resolved (was: Patch Available)
Issue resolved by pull request 1572
[https://github.com/apache/kafka/pull/1572]
> Subsequent regex subscription calls fail
>
>
> Key: KAFKA-3854
> URL: https://issues.apache.org/jira/browse/KAFKA-3854
> Project: Kafka
> Issue Type: Bug
> Components: consumer
>Reporter: Vahid Hashemian
>Assignee: Vahid Hashemian
> Fix For: 0.10.1.0
>
>
> There are a couple of issues with regex subscription in the new consumer:
> h6. Subsequent Subscriptions Fail
> When consecutive calls are made to new consumer's [regex
> subscription|https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java#L850],
> like below:
> {code}
> consumer.subscribe(Pattern.compile("..."), listener);
> consumer.poll(0);
> consumer.subscribe(Pattern.compile("f.."), listener);
> consumer.poll(0);
> {code}
> the second call fails with the following error:
> {code}
> Exception in thread "main" java.lang.IllegalStateException: Subscription to
> topics, partitions and pattern are mutually exclusive
> at
> org.apache.kafka.clients.consumer.internals.SubscriptionState.subscribe(SubscriptionState.java:175)
> at
> org.apache.kafka.clients.consumer.KafkaConsumer.subscribe(KafkaConsumer.java:854)
> ...
> {code}
> h6. Subsequent Subscriptions Fail to Trigger a Subscription Change and
> Rebalance
> Even after the code is tweaked to get around the above issue, only the first
> call to regex subscription triggers an actual subscription and the
> subscription's rebalance listener. The reason is the regex {{subscription}} /
> {{poll}} does not directly call {{changeSubscription(...)}} in which
> {{needsPartitionAssignment}} is set and causes a rebalance. This method is
> called only during the first regex {{subscription}} / {{poll}} when
> coordinator is unknown and [a {{client.poll}}
> call|https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractCoordinator.java#L179]
> is made which eventually leads to [a {{changeSubscription}}
> call|https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java#L161].
> The subsequent call does not reach this point because the coordinator is
> already known.
> It seems due to the same reason, if a consumer is subscribed to a pattern,
> and later on a new topic is created that matches that pattern, the consumer
> does not become subscribed it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (KAFKA-3854) Subsequent regex subscription calls fail
[
https://issues.apache.org/jira/browse/KAFKA-3854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vahid Hashemian updated KAFKA-3854:
---
Status: Patch Available (was: In Progress)
> Subsequent regex subscription calls fail
>
>
> Key: KAFKA-3854
> URL: https://issues.apache.org/jira/browse/KAFKA-3854
> Project: Kafka
> Issue Type: Bug
> Components: consumer
>Reporter: Vahid Hashemian
>Assignee: Vahid Hashemian
>
> There are a couple of issues with regex subscription in the new consumer:
> h6. Subsequent Subscriptions Fail
> When consecutive calls are made to new consumer's [regex
> subscription|https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java#L850],
> like below:
> {code}
> consumer.subscribe(Pattern.compile("..."), listener);
> consumer.poll(0);
> consumer.subscribe(Pattern.compile("f.."), listener);
> consumer.poll(0);
> {code}
> the second call fails with the following error:
> {code}
> Exception in thread "main" java.lang.IllegalStateException: Subscription to
> topics, partitions and pattern are mutually exclusive
> at
> org.apache.kafka.clients.consumer.internals.SubscriptionState.subscribe(SubscriptionState.java:175)
> at
> org.apache.kafka.clients.consumer.KafkaConsumer.subscribe(KafkaConsumer.java:854)
> ...
> {code}
> h6. Subsequent Subscriptions Fail to Trigger a Subscription Change and
> Rebalance
> Even after the code is tweaked to get around the above issue, only the first
> call to regex subscription triggers an actual subscription and the
> subscription's rebalance listener. The reason is the regex {{subscription}} /
> {{poll}} does not directly call {{changeSubscription(...)}} in which
> {{needsPartitionAssignment}} is set and causes a rebalance. This method is
> called only during the first regex {{subscription}} / {{poll}} when
> coordinator is unknown and [a {{client.poll}}
> call|https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractCoordinator.java#L179]
> is made which eventually leads to [a {{changeSubscription}}
> call|https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java#L161].
> The subsequent call does not reach this point because the coordinator is
> already known.
> It seems due to the same reason, if a consumer is subscribed to a pattern,
> and later on a new topic is created that matches that pattern, the consumer
> does not become subscribed it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (KAFKA-3854) Subsequent regex subscription calls fail
[
https://issues.apache.org/jira/browse/KAFKA-3854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vahid Hashemian updated KAFKA-3854:
---
Description:
There are a couple of issues with regex subscription in the new consumer:
h6. Subsequent Subscriptions Fail
When consecutive calls are made to new consumer's [regex
subscription|https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java#L850],
like below:
{code}
consumer.subscribe(Pattern.compile("..."), listener);
consumer.poll(0);
consumer.subscribe(Pattern.compile("f.."), listener);
consumer.poll(0);
{code}
the second call fails with the following error:
{code}
Exception in thread "main" java.lang.IllegalStateException: Subscription to
topics, partitions and pattern are mutually exclusive
at
org.apache.kafka.clients.consumer.internals.SubscriptionState.subscribe(SubscriptionState.java:175)
at
org.apache.kafka.clients.consumer.KafkaConsumer.subscribe(KafkaConsumer.java:854)
...
{code}
h6. Subsequent Subscriptions Fail to Trigger a Subscription Change and Rebalance
Even after the code is tweaked to get around the above issue, only the first
call to regex subscription triggers an actual subscription and the
subscription's rebalance listener. The reason is the regex {{subscription}} /
{{poll}} does not directly call {{changeSubscription(...)}} in which
{{needsPartitionAssignment}} is set and causes a rebalance. This method is
called only during the first regex {{subscription}} / {{poll}} when coordinator
is unknown and [a {{client.poll}}
call|https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractCoordinator.java#L179]
is made which eventually leads to [a {{changeSubscription}}
call|https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java#L161].
The subsequent call does not reach this point because the coordinator is
already known.
It seems due to the same reason, if a consumer is subscribed to a pattern, and
later on a new topic is created that matches that pattern, the consumer does
not become subscribed it.
was:
There are a couple of issues with regex subscription in the new consumer:
h6. Subsequent Subscriptions Fail
When consecutive calls are made to new consumer's [regex
subscription|https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java#L850],
like below:
{code}
consumer.subscribe(Pattern.compile("..."), listener);
consumer.poll(0);
consumer.subscribe(Pattern.compile("f.."), listener);
consumer.poll(0);
{code}
the second call fails with the following error:
{code}
Exception in thread "main" java.lang.IllegalStateException: Subscription to
topics, partitions and pattern are mutually exclusive
at
org.apache.kafka.clients.consumer.internals.SubscriptionState.subscribe(SubscriptionState.java:175)
at
org.apache.kafka.clients.consumer.KafkaConsumer.subscribe(KafkaConsumer.java:854)
...
{code}
h6. Subsequent Subscriptions Fail to Trigger a Subscription Change and Rebalance
Even after the code is tweaked to get around the above issue, only the first
call to regex subscription triggers an actual subscription and the
subscription's rebalance listener. The reason is the regex {{subscription}} /
{{poll}} does not directly call {{changeSubscription(...)}} in which
{{needsPartitionAssignment}} is set and causes a rebalance. This method is
called only during the first regex {{subscription}} / {{poll}} when coordinator
is unknown and [a {{client.poll}}
call|https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractCoordinator.java#L179]
is made which eventually leads to [a {{changeSubscription}}
call|https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java#L161].
The subsequent call does not reach this point because the coordinator is
already known.
> Subsequent regex subscription calls fail
>
>
> Key: KAFKA-3854
> URL: https://issues.apache.org/jira/browse/KAFKA-3854
> Project: Kafka
> Issue Type: Bug
> Components: consumer
>Reporter: Vahid Hashemian
>Assignee: Vahid Hashemian
>
> There are a couple of issues with regex subscription in the new consumer:
> h6. Subsequent Subscriptions Fail
> When consecutive calls are made to new consumer's [regex
> subscription|https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java#L850],
> like below:
> {code}
> consumer.subscribe(Pattern.compile("..."), listener);
> consumer.poll(0);
>
[jira] [Updated] (KAFKA-3854) Subsequent regex subscription calls fail
[
https://issues.apache.org/jira/browse/KAFKA-3854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vahid Hashemian updated KAFKA-3854:
---
Description:
There are a couple of issues with regex subscription in the new consumer:
h6. Subsequent Subscriptions Fail
When consecutive calls are made to new consumer's [regex
subscription|https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java#L850],
like below:
{code}
consumer.subscribe(Pattern.compile("..."), listener);
consumer.poll(0);
consumer.subscribe(Pattern.compile("f.."), listener);
consumer.poll(0);
{code}
the second call fails with the following error:
{code}
Exception in thread "main" java.lang.IllegalStateException: Subscription to
topics, partitions and pattern are mutually exclusive
at
org.apache.kafka.clients.consumer.internals.SubscriptionState.subscribe(SubscriptionState.java:175)
at
org.apache.kafka.clients.consumer.KafkaConsumer.subscribe(KafkaConsumer.java:854)
...
{code}
h6. Subsequent Subscriptions Fail to Trigger a Subscription Change and Rebalance
Even after the code is tweaked to get around the above issue, only the first
call to regex subscription triggers an actual subscription and the
subscription's rebalance listener. The reason is the regex {{subscription}} /
{{poll}} does not directly call {{changeSubscription(...)}} in which
{{needsPartitionAssignment}} is set and causes a rebalance. This method is
called only during the first regex {{subscription}} / {{poll}} when coordinator
is unknown and [a {{client.poll}}
call|https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractCoordinator.java#L179]
is made which eventually leads to [a {{changeSubscription}}
call|https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java#L161].
The subsequent call does not reach this point because the coordinator is
already known.
was:
There are a couple of issues with regex subscription in the new consumer:
h6. Subsequent Subscriptions Fail
When consecutive calls are made to new consumer's [regex
subscription|https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java#L850],
like below:
{code}
consumer.subscribe(Pattern.compile("..."), listener);
consumer.poll(0);
consumer.subscribe(Pattern.compile("f.."), listener);
consumer.poll(0);
{code}
the second call fails with the following error:
{code}
Exception in thread "main" java.lang.IllegalStateException: Subscription to
topics, partitions and pattern are mutually exclusive
at
org.apache.kafka.clients.consumer.internals.SubscriptionState.subscribe(SubscriptionState.java:175)
at
org.apache.kafka.clients.consumer.KafkaConsumer.subscribe(KafkaConsumer.java:854)
...
{code}
h6. Subsequent Subscriptions Fail to Trigger a Rebalance
Even after the code is tweaked to get around the above issue, only the first
call to regex subscription triggers the subscription's rebalance listener. The
reason is the regex {{subscription}} / {{poll}} does not directly call
{{changeSubscription(...)}} in which {{needsPartitionAssignment}} is set and
causes a rebalance. This method is called only during the first regex
{{subscription}} / {{poll}} when coordinator is unknown and [a {{client.poll}}
call|https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractCoordinator.java#L179]
is made which eventually leads to [a {{changeSubscription}}
call|https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java#L161].
The subsequent call does not reach this point because the coordinator is
already known.
> Subsequent regex subscription calls fail
>
>
> Key: KAFKA-3854
> URL: https://issues.apache.org/jira/browse/KAFKA-3854
> Project: Kafka
> Issue Type: Bug
> Components: consumer
>Reporter: Vahid Hashemian
>Assignee: Vahid Hashemian
>
> There are a couple of issues with regex subscription in the new consumer:
> h6. Subsequent Subscriptions Fail
> When consecutive calls are made to new consumer's [regex
> subscription|https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java#L850],
> like below:
> {code}
> consumer.subscribe(Pattern.compile("..."), listener);
> consumer.poll(0);
> consumer.subscribe(Pattern.compile("f.."), listener);
> consumer.poll(0);
> {code}
> the second call fails with the following error:
> {code}
> Exception in thread "main" java.lang.IllegalStateException: Subscription to
> topics,
[jira] [Updated] (KAFKA-3854) Subsequent regex subscription calls fail
[
https://issues.apache.org/jira/browse/KAFKA-3854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vahid Hashemian updated KAFKA-3854:
---
Summary: Subsequent regex subscription calls fail (was: Consecutive regex
subscription calls fail)
> Subsequent regex subscription calls fail
>
>
> Key: KAFKA-3854
> URL: https://issues.apache.org/jira/browse/KAFKA-3854
> Project: Kafka
> Issue Type: Bug
> Components: consumer
>Reporter: Vahid Hashemian
>Assignee: Vahid Hashemian
>
> There are a couple of issues with regex subscription in the new consumer:
> h6. Subsequent Subscriptions Fail
> When consecutive calls are made to new consumer's [regex
> subscription|https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java#L850],
> like below:
> {code}
> consumer.subscribe(Pattern.compile("..."), listener);
> consumer.poll(0);
> consumer.subscribe(Pattern.compile("f.."), listener);
> consumer.poll(0);
> {code}
> the second call fails with the following error:
> {code}
> Exception in thread "main" java.lang.IllegalStateException: Subscription to
> topics, partitions and pattern are mutually exclusive
> at
> org.apache.kafka.clients.consumer.internals.SubscriptionState.subscribe(SubscriptionState.java:175)
> at
> org.apache.kafka.clients.consumer.KafkaConsumer.subscribe(KafkaConsumer.java:854)
> ...
> {code}
> h6. Subsequent Subscriptions Fail to Trigger a Rebalance
> Even after the code is tweaked to get around the above issue, only the first
> call to regex subscription triggers the subscription's rebalance listener.
> The reason is the regex {{subscription}} / {{poll}} does not directly call
> {{changeSubscription(...)}} in which {{needsPartitionAssignment}} is set and
> causes a rebalance. This method is called only during the first regex
> {{subscription}} / {{poll}} when coordinator is unknown and [a
> {{client.poll}}
> call|https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractCoordinator.java#L179]
> is made which eventually leads to [a {{changeSubscription}}
> call|https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java#L161].
> The subsequent call does not reach this point because the coordinator is
> already known.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
