[ https://issues.apache.org/jira/browse/SPARK-26150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16767617#comment-16767617 ]
Marcelo Vanzin commented on SPARK-26150: ---------------------------------------- If you can provide the full YARN logs for your application that would help, but otherwise, it's hard to know what exactly is causing the problem here. > __spark_conf__XXX.zip doesn't exist > ----------------------------------- > > Key: SPARK-26150 > URL: https://issues.apache.org/jira/browse/SPARK-26150 > Project: Spark > Issue Type: Bug > Components: YARN > Affects Versions: 1.6.0 > Reporter: Madhusoodan > Priority: Major > > Spark application submitted via spark-submit fails because the resource > localization for the application master instance 2 fails. > > This is the scenario. > # I submitted an application via spark-submit > # The application master failed (The reason is not important here, but here > it is: jar was compiled through java 8 and the java installed in cluster > nodes was java 7) > # The appmaster instance 1 deleted the staging directory on failure. (Below > is the log excerpt) > ## Deleting staging directory .sparkStaging/application_1542037527280_0015 > # The Yarn restarted the appmaster in another node, where the NM couldn't > find the resource in HDFS resulting in to (from nodemanager logs) > ## Diagnostics: File does not exist: > hdfs://mpcdh001.informatica.com:8020/user/sampleuser/.sparkStaging/application_1542037527280_0007/__spark_conf__7096300976806305459.zip > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org