Re: Spark job hangs when History server events are written to hdfs

Regards Pankaj From: Archit Thakur mailto:archit279tha...@gmail.com>> Date: Wednesday, 8 July 2015 4:36 pm To: Akhil Das mailto:ak...@sigmoidanalytics.com>> Cc: Pankaj Arora mailto:pankaj.ar...@guavus.com>>, "dev@spark.apache.org<mailto:dev@spark.apache.org>" mai

Spark job hangs when History server events are written to hdfs

Hi, I am running long running application over yarn using spark and I am facing issues while using spark’s history server when the events are written to hdfs. It seems to work fine for some time and in between I see following exception. 2015-06-01 00:00:03,247 [SparkListenerBus] ERROR org.apa