Hello, I'm getting the below exception when testing Spark 2.0 with Kafka 0.10.
16/08/23 16:31:01 INFO AppInfoParser: Kafka version : 0.10.0.0 > 16/08/23 16:31:01 INFO AppInfoParser: Kafka commitId : b8642491e78c5a13 > 16/08/23 16:31:01 INFO CachedKafkaConsumer: Initial fetch for > spark-executor-example mt_event 0 15782114 > 16/08/23 16:31:01 INFO AbstractCoordinator: Discovered coordinator > 10.150.254.161:9233 (id: 2147483646 rack: null) for group > spark-executor-example. > 16/08/23 16:31:02 ERROR Executor: Exception in task 0.0 in stage 1.0 (TID > 6) > java.lang.AssertionError: assertion failed: Failed to get records for > spark-executor-example mt_event 0 15782114 after polling for 512 > at scala.Predef$.assert(Predef.scala:170) > at > org.apache.spark.streaming.kafka010.CachedKafkaConsumer.get(CachedKafkaConsumer.scala:74) > at > org.apache.spark.streaming.kafka010.KafkaRDD$KafkaRDDIterator.next(KafkaRDD.scala:227) > at > org.apache.spark.streaming.kafka010.KafkaRDD$KafkaRDDIterator.next(KafkaRDD.scala:193) > at scala.collection.Iterator$$anon$11.next(Iterator.scala:409) > I get this error intermittently. Sometimes a few batches are scheduled and run fine. Then I get this error. kafkacat is able to fetch from this topic continuously. Full exception is here -- https://gist.github.com/SrikanthTati/c2e95c4ac689cd49aab817e24ec42767 Srikanth
