we are using spark kafka streaming. We have 6 nodes in kafka cluster if any of the node is getting down we are getting below exception and streaming stopped. ERROR DirectKafkaInputDStream:70 - ArrayBuffer(kafka.common.NotLeaderForPartitionException, kafka.common.NotLeaderForPartitionException, org.apache.spark.SparkException: Couldn't find leader offsets for Set([techops-prod2,4], [techops-prod2,0]))
Please let me know do we missed any setting so that streaming should not stopped even if couple of Kafka nodes are down. Thanks Amit
