Adam Balogh created SPARK-25200:
-----------------------------------
Summary: Allow setting HADOOP_CONF_DIR as a spark property
Key: SPARK-25200
URL: https://issues.apache.org/jira/browse/SPARK-25200
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 2.3.1
Reporter: Adam Balogh
When submitting applications to Yarn in cluster mode, using the
InProcessLauncher, spark finds the cluster's configuration files based on the
HADOOP_CONF_DIR environment variable. This does not make it possible to submit
to more than one Yarn clusters concurrently using the InProcessLauncher.
I think we should make it possible to define HADOOP_CONF_DIR as a spark
property, so it can be different for each spark submission.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
