[ 
https://issues.apache.org/jira/browse/HADOOP-18495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616920#comment-17616920
 ] 

Steve Loughran commented on HADOOP-18495:
-----------------------------------------

it is there, it works, except when env var expansion has been disabled for 
security reasons, or are running on a host where the env var is unset., just as 
you can in bash. 

you can provide fallback values in either situation, ${env.VAR:-fallback}

regarding the two reported failure situations
* accessing configuration throught http://namenode/conf : config is dumped 
without variable expansion, and with redaction of known secrets. we dont' want 
env.AWS_SECRET_KEY or fs.s3a.secret.key leaking, do we?
* spark. make sure the env var is being expanded in the host/process with it 
set. writing a spark RDD to evaluate env vars in the workers is straightforward 
if you want to explore this inpendendly.

closing as WORKSFORME. 

> Configuration Resolver not working on api
> -----------------------------------------
>
>                 Key: HADOOP-18495
>                 URL: https://issues.apache.org/jira/browse/HADOOP-18495
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: common
>    Affects Versions: 3.3.4
>         Environment: Hadoop 3.3.4 + Spark 3.3.0
>            Reporter: Francesco Scipioni
>            Priority: Major
>
> - Following [Configuration 
> javadoc|https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java]
> - Using property value as 
> {code:xml}
> ${env.ENV_VARIABLE} 
> {code}
> - Cluster Startup well, and property is correctly replaced
> Launching a Spark Job or accessing configuration throught 
> http://namenode/conf property value is not replaced and stile provided as 
> value ${env.ENV_VARIABLE}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to