Could you please describe the version of Spark, and how did you run your app? If you don’t mind to share minimal app which can reproduce this, it would be really great.
- Jungtaek Lim (HeartSaVioR) On Mon, 2 Jul 2018 at 7:56 PM kant kodali <kanth...@gmail.com> wrote: > Hi All, > > I get the below error quite often when I do an stream-stream inner join on > two data frames. After running through several experiments stream-stream > joins dont look stable enough for production yet. any advice on this? > > Thanks! > > java.util.ConcurrentModificationException: KafkaConsumer is not safe for > multi-threaded access > 18/07/02 09:32:14 INFO LineBufferedStream: stdout: at > org.apache.kafka.clients.consumer.KafkaConsumer.acquire(KafkaConsumer.java:1431) > 18/07/02 09:32:14 INFO LineBufferedStream: stdout: at > org.apache.kafka.clients.consumer.KafkaConsumer.close(KafkaConsumer.java:1361) > 18/07/02 09:32:14 INFO LineBufferedStream: stdout: at > org.apache.spark.sql.kafka010.CachedKafkaConsumer.close(CachedKafkaConsumer.scala:301) >
