Hi, I had a similar issue using 2.1.0 but not with Kafka. Updating to 2.1.1 solved my issue. Can you try with 2.1.1 as well and report back?
Best, Anastasios Am 17.09.2017 16:48 schrieb "HARSH TAKKAR" <takkarha...@gmail.com>: Hi I am using spark 2.1.0 with scala 2.11.8, and while iterating over the partitions of each rdd in a dStream formed using KafkaUtils, i am getting the below exception, please suggest a fix. I have following config kafka : enable.auto.commit:"true", auto.commit.interval.ms:"1000", session.timeout.ms:"30000", Spark: spark.streaming.backpressure.enabled=true spark.streaming.kafka.maxRatePerPartition=200 Exception in task 0.2 in stage 3236.0 (TID 77795) java.util.ConcurrentModificationException: KafkaConsumer is not safe for multi-threaded access -- Kind Regards Harsh
