Terence Yim created YARN-4727: --------------------------------- Summary: Unable to override the $HADOOP_CONF_DIR env variable for container Key: YARN-4727 URL: https://issues.apache.org/jira/browse/YARN-4727 Project: Hadoop YARN Issue Type: Bug Components: nodemanager Affects Versions: 2.6.4, 2.7.2, 2.5.2, 2.4.1 Reporter: Terence Yim
Given the default config of "yarn.nodemanager.env-whitelist", application should be able to set the env variable $HADOOP_CONF_DIR to value other than the one in the NodeManager system environment. However, I believe due to a bug in the {{org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch}} class, it is not possible so. >From the {{sanitizeEnv()}} method in the ContainerLaunch class >(https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerLaunch.java#L991) {noformat} String[] whitelist = conf.get(YarnConfiguration.NM_ENV_WHITELIST, YarnConfiguration.DEFAULT_NM_ENV_WHITELIST).split(","); for(String whitelistEnvVariable : whitelist) { putEnvIfAbsent(environment, whitelistEnvVariable.trim()); } ... private static void putEnvIfAbsent( Map<String, String> environment, String variable) { if (environment.get(variable) == null) { putEnvIfNotNull(environment, variable, System.getenv(variable)); } } {noformat} The {{putEnvIfAbsent}} call simply use the {{whitelistEnvVariable}} to call {{System.getenv(whitelistEnvVariable)}} and set it to the environment map. It should be using the one from the {{launchContext}} instead. -- This message was sent by Atlassian JIRA (v6.3.4#6332)