[
https://issues.apache.org/jira/browse/SPARK-25200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16716847#comment-16716847
]
ASF GitHub Bot commented on SPARK-25200:
----------------------------------------
adambalogh commented on issue #22289: [SPARK-25200][YARN] Allow specifying
HADOOP_CONF_DIR as spark property
URL: https://github.com/apache/spark/pull/22289#issuecomment-446164710
Sorry for the lack of response, we found that adding the right files to the
inprocess spark-submit's classpath mostly does the job, so I think I'll leave
this as it is now.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Allow setting HADOOP_CONF_DIR as a spark property
> -------------------------------------------------
>
> Key: SPARK-25200
> URL: https://issues.apache.org/jira/browse/SPARK-25200
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 2.3.1
> Reporter: Adam Balogh
> Priority: Major
>
> When submitting applications to Yarn in cluster mode, using the
> InProcessLauncher, spark finds the cluster's configuration files based on the
> HADOOP_CONF_DIR environment variable. This does not make it possible to
> submit to more than one Yarn clusters concurrently using the
> InProcessLauncher.
> I think we should make it possible to define HADOOP_CONF_DIR as a spark
> property, so it can be different for each spark submission.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
