[ https://issues.apache.org/jira/browse/SPARK-1870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tathagata Das updated SPARK-1870: --------------------------------- Affects Version/s: (was: 1.0.0) > Jars specified via --jars in spark-submit are not added to executor classpath > for YARN > -------------------------------------------------------------------------------------- > > Key: SPARK-1870 > URL: https://issues.apache.org/jira/browse/SPARK-1870 > Project: Spark > Issue Type: Bug > Components: YARN > Reporter: Xiangrui Meng > Priority: Critical > Fix For: 1.0.0 > > > With `spark-submit`, jars specified via `--jars` are added to distributed > cache in `yarn-cluster` mode. The executor should add cached jars to > classpath. However, > {code} > sc.parallelize(0 to 10, 10).map { i => > System.getProperty("java.class.path") > }.collect().foreach(println) > {code} > shows only system jars, `app.jar`, and `spark.jar` but not other jars in the > distributed cache. > The workaround is using assembly jar. -- This message was sent by Atlassian JIRA (v6.2#6252)