Re: Not all KafkaReceivers processing the data Why?

: 88 > > Topic: CEQReceiver Partition: 4Leader: 89 Replicas: > 89Isr: 89 > > > > *From:* Jeff Nadler [mailto:jnad...@srcginc.com] > *Sent:* Wednesday, September 14, 2016 12:46 PM > *To:* Rachana Srivastava > *Cc:* user@spark.apache.org; d...@spark

Re: Not all KafkaReceivers processing the data Why?

Take a look at how the messages are actually distributed across the partitions. If the message keys have a low cardinality, you might get poor distribution (i.e. all the messages are actually only in two of the five partitions, leading to what you see in Spark). If you take a look at the Kafka

Re: Not all KafkaReceivers processing the data Why?

Have you checked your Kafka brokers to be certain that data is going to all 5 partitions?We use something very similar (but in Scala) and have no problems. Also you might not get the best response blasting both user+dev lists like this. Normally you'd want to use 'user' only. -Jeff On

Not all KafkaReceivers processing the data Why?

Hello all, I have created a Kafka topic with 5 partitions. And I am using createStream receiver API like following. But somehow only one receiver is getting the input data. Rest of receivers are not processign anything. Can you please help? JavaPairDStream messages = null;