Have you checked your Kafka brokers to be certain that data is going to all 5 partitions? We use something very similar (but in Scala) and have no problems.
Also you might not get the best response blasting both user+dev lists like this. Normally you'd want to use 'user' only. -Jeff On Wed, Sep 14, 2016 at 12:33 PM, Rachana Srivastava < rachana.srivast...@markmonitor.com> wrote: > Hello all, > > > > I have created a Kafka topic with 5 partitions. And I am using > createStream receiver API like following. But somehow only one receiver > is getting the input data. Rest of receivers are not processign anything. > Can you please help? > > > > JavaPairDStream<String, String> messages = null; > > > > if(sparkStreamCount > 0){ > > // We create an input DStream for each partition of the > topic, unify those streams, and then repartition the unified stream. > > List<JavaPairDStream<String, String>> kafkaStreams = new > ArrayList<JavaPairDStream<String, String>>(sparkStreamCount); > > for (int i = 0; i < sparkStreamCount; i++) { > > kafkaStreams.add( > KafkaUtils.createStream(jssc, contextVal.getString(KAFKA_ZOOKEEPER), > contextVal.getString(KAFKA_GROUP_ID), kafkaTopicMap)); > > } > > messages = jssc.union(kafkaStreams.get(0), > kafkaStreams.subList(1, kafkaStreams.size())); > > } > > else{ > > messages = KafkaUtils.createStream(jssc, > contextVal.getString(KAFKA_ZOOKEEPER), contextVal.getString(KAFKA_GROUP_ID), > kafkaTopicMap); > > } > > > > > > > > > > > > > > >