[
https://issues.apache.org/jira/browse/FLINK-11135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paul Lin updated FLINK-11135:
-----------------------------
Description: Currently, Hadoop config search order is: `HADOOP_HOME >
HADOOP_CONF_DIR > Flink configuration entries`. However, YARN client perfers
`HADOOP_CONF_DIR` to `HADOOP_HOME`, so it would result in inconsistency if the
two env variables point to different paths. Hence I propose to refactor the
Hadoop config loading in HadoopUtils, changing the search order to
`HADOOP_CONF_DIR > Flink configuration entries > HADOOP_HOME`. (was: In
[FLINK-7967] we mark the Hadoop specific Flink configuration options as
deprecated and recommend users to use HADOOP_CONF_DIR env variable instead, but
if both are configured, Flink would still look for Hadoop configurations in the
path specified in Flink configurations first. Plus, the code that searches
Hadoop configurations in `fs.hdfs.hadoopconf` is wrongly placed in the env
variables section.
I think the search order should be: HADOOP_CONF_DIR > Flink conf options >
HADOOP_HOME.)
Summary: Refactor Hadoop config loading orders (was: Reduce priority
of the deprecated Hadoop specific Flink conf options)
> Refactor Hadoop config loading orders
> -------------------------------------
>
> Key: FLINK-11135
> URL: https://issues.apache.org/jira/browse/FLINK-11135
> Project: Flink
> Issue Type: Improvement
> Components: YARN
> Affects Versions: 1.6.2, 1.7.0
> Reporter: Paul Lin
> Assignee: Paul Lin
> Priority: Minor
>
> Currently, Hadoop config search order is: `HADOOP_HOME > HADOOP_CONF_DIR >
> Flink configuration entries`. However, YARN client perfers `HADOOP_CONF_DIR`
> to `HADOOP_HOME`, so it would result in inconsistency if the two env
> variables point to different paths. Hence I propose to refactor the Hadoop
> config loading in HadoopUtils, changing the search order to `HADOOP_CONF_DIR
> > Flink configuration entries > HADOOP_HOME`.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
