[
https://issues.apache.org/jira/browse/KAFKA-4269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15563913#comment-15563913
]
ASF GitHub Bot commented on KAFKA-4269:
---------------------------------------
GitHub user bbejeck opened a pull request:
https://github.com/apache/kafka/pull/2005
KAFKA-4269 extracted code updating topics when regex pattern specifie…
…d out of topicGroups method. The topicGroups method only called from
StreamPartitionAssignor when KafkaStreams object is the leader, needs to be
executed for clients.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/bbejeck/kafka
KAFKA-4269_multiple_kstream_instances_mult_consumers_npe
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/kafka/pull/2005.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2005
----
commit 58c694e14fd4ba5ee134be54e3f75f4aafa8a1d6
Author: bbejeck <bbej...@gmail.com>
Date: 2016-10-10T23:26:24Z
KAFKA-4269 extracted code updating topics when regex pattern specified out
of topicGroups method. The topicGroups method only called from
StreamPartitionAssignor when KafkaStreams object is the leader, needs to be
executed for clients.
----
> Multiple KStream instances with at least one Regex source causes NPE when
> using multiple consumers
> --------------------------------------------------------------------------------------------------
>
> Key: KAFKA-4269
> URL: https://issues.apache.org/jira/browse/KAFKA-4269
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 0.10.1.0
> Reporter: Bill Bejeck
> Assignee: Bill Bejeck
>
> I discovered this issue while doing testing for for KAFKA-4114.
> KAFKA-4131 fixed the issue of a _single_ KStream with a regex source on
> partitioned topics across multiple consumers.
> //KAFKA-4131 fixed this case assuming an "foo*" topics are partitioned
> KStream kstream = builder.source(Pattern.compile("foo.*"));
> KafkaStream stream = new KafkaStreams(builder, props);
> stream.start();
> This is a new issue where there are _multiple_
> KStream instances (and one has a regex source) within a single KafkaStreams
> object. When running the second or "following"
> consumer there are NPE errors generated in the RecordQueue.addRawRecords
> method when attempting to consume records.
> For example:
> KStream kstream = builder.source(Pattern.compile("foo.*"));
> KStream kstream2 = builder.source(.....): //can be regex or named topic
> sources
> KafkaStream stream = new KafkaStreams(builder, props);
> stream.start();
> By adding an additional KStream instance like above (whether Regex or Named
> topic) causes a NPE when run as "follower"
> From my initial debugging I can see the TopicPartition assignments being set
> on the "follower" KafkaStreams instance, but need to track down why and where
> all assignments aren't being set.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
