[
https://issues.apache.org/jira/browse/SPARK-21377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-21377:
------------------------------------
Assignee: Apache Spark
> Add a new configuration to extend AM classpath in yarn client mode
> ------------------------------------------------------------------
>
> Key: SPARK-21377
> URL: https://issues.apache.org/jira/browse/SPARK-21377
> Project: Spark
> Issue Type: Bug
> Components: YARN
> Affects Versions: 2.2.0
> Reporter: Yesha Vora
> Assignee: Apache Spark
> Priority: Minor
>
> In this issue we have a long running Spark application with secure HBase,
> which requires {{HBaseCredentialProvider}} to get tokens periodically, we
> specify HBase related jars with {{\--packages}}, but these dependencies are
> not added into AM classpath, so when {{HBaseCredentialProvider}} tries to
> initialize HBase connections to get tokens, it will be failed.
> Currently because jars specified with {{\--jars}} or {{\--packages}} are not
> added into AM classpath, the only way to extend AM classpath is to use
> "spark.driver.extraClassPath" which supposed to be used in yarn cluster mode.
> So here we should figure out a solution either to put these dependencies to
> AM classpath or to extend AM classpath with correct configuration.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
