[ https://issues.apache.org/jira/browse/SPARK-21377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Saisai Shao updated SPARK-21377: -------------------------------- Summary: Jars specified with --jars or --packages are not added into AM's system classpath (was: Add a new configuration to extend AM classpath in yarn client mode) > Jars specified with --jars or --packages are not added into AM's system > classpath > --------------------------------------------------------------------------------- > > Key: SPARK-21377 > URL: https://issues.apache.org/jira/browse/SPARK-21377 > Project: Spark > Issue Type: Bug > Components: YARN > Affects Versions: 2.2.0 > Reporter: Yesha Vora > Priority: Minor > > In this issue we have a long running Spark application with secure HBase, > which requires {{HBaseCredentialProvider}} to get tokens periodically, we > specify HBase related jars with {{\--packages}}, but these dependencies are > not added into AM classpath, so when {{HBaseCredentialProvider}} tries to > initialize HBase connections to get tokens, it will be failed. > Currently because jars specified with {{\--jars}} or {{\--packages}} are not > added into AM classpath, the only way to extend AM classpath is to use > "spark.driver.extraClassPath" which supposed to be used in yarn cluster mode. > So here we should figure out a solution either to put these dependencies to > AM classpath or to extend AM classpath with correct configuration. -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org