Re: Spark (1.2.0) submit fails with exception saying log directory already exists

This probably means your app is failing and the second attempt is hitting that issue. You may fix the directory already exists error by setting spark.eventLog.overwrite=true in your conf, but most probably that will just expose the actual error in your app. On Tue, Aug 25, 2015 at 9:37 AM,

Spark (1.2.0) submit fails with exception saying log directory already exists

Here is the error yarn.ApplicationMaster: Final app status: FAILED, exitCode: 15, (reason: User class threw exception: Log directory hdfs://Sandbox/user/spark/applicationHistory/application_1438113296105_0302 already exists!) I am using cloudera 5.3.2 with Spark 1.2.0 Any help is appreciated.