[
https://issues.apache.org/jira/browse/KAFKA-4131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15478891#comment-15478891
]
ASF GitHub Bot commented on KAFKA-4131:
---------------------------------------
GitHub user bbejeck opened a pull request:
https://github.com/apache/kafka/pull/1843
KAFKA-4131: moved updates for topics matching regex subscription meth…
…od, allows for followers to get subscribed topics
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/bbejeck/kafka
KAFKA-4131_mulitple_regex_consumers_cause_npe
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/kafka/pull/1843.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1843
----
commit 0816ecc454cb9d7504fbc2c48549e3f4eb2b5f22
Author: bbejeck <bbej...@gmail.com>
Date: 2016-09-10T01:40:12Z
KAFKA-4131: moved updates for topics matching regex subscription method,
allows for followers to get subscribed topics
----
> Multiple Regex KStream-Consumers cause Null pointer exception in
> addRawRecords in RecordQueue class
> ---------------------------------------------------------------------------------------------------
>
> Key: KAFKA-4131
> URL: https://issues.apache.org/jira/browse/KAFKA-4131
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 0.10.1.0
> Environment: Servers: Confluent Distribution 3.0.0 (i.e. kafka 0.10.0
> release)
> Client: Kafka-streams and Kafka-client... commit:
> 6fb33afff976e467bfa8e0b29eb827
> 70a2a3aaec
> Reporter: David J. Garcia
> Assignee: Bill Bejeck
>
> When you start two consumer processes with a regex topic (with 2 or more
> partitions for the matching topics), the second (i.e. nonleader) consumer
> will fail with a null pointer exception.
> Exception in thread "StreamThread-4" java.lang.NullPointerException
> at org.apache.kafka.streams.processor.internals.
> RecordQueue.addRawRecords(RecordQueue.java:78)
> at org.apache.kafka.streams.processor.internals.
> PartitionGroup.addRawRecords(PartitionGroup.java:117)
> at org.apache.kafka.streams.processor.internals.
> StreamTask.addRecords(StreamTask.java:139)
> at org.apache.kafka.streams.processor.internals.
> StreamThread.runLoop(StreamThread.java:299)
> at org.apache.kafka.streams.processor.internals.
> StreamThread.run(StreamThread.java:208)
> The issue may be in the TopologyBuilder line 832:
> String[] topics = (sourceNodeFactory.pattern != null) ?
> sourceNodeFactory.getTopics(subscriptionUpdates.getUpdates()) :
> sourceNodeFactory.getTopics();
> Because the 2nd consumer joins as a follower, “getUpdates” returns an
> empty collection and the regular expression doesn’t get applied to any
> topics.
> Steps to Reproduce:
> 1.) Create at least two topics with at least 2 partitions each. And start
> sending messages to them.
> 2.) Start a single threaded Regex KStream-Consumer (i.e. becomes the leader)
> 3) Start a new instance of this consumer (i.e. it should receive some of the
> partitions)
> The second consumer will die with the above exception.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
