[ https://issues.apache.org/jira/browse/MAPREDUCE-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15677579#comment-15677579 ]
Junping Du commented on MAPREDUCE-6565: --------------------------------------- bq. I think the tarball is muddying the waters here. *-site.xml files should be treated the same whether they are in a tarball or not. It'd be tricky and messy to do otherwise. Essentially what we're asking is whether clients should be able to override any non-final setting in the *-site.xml files with their job.xml setting, even if that setting is a "server side" property Agree. Whether clients are allowed to override any non-final setting is something I originally asking for. But later, I realized that if we disallow client to override "server-setting" - like case here, tar-ball configuration will become the final setting for "server-side" configuration. It also means MR tarball will become various according to cluster settings but not according to releases. Standing at support-ability prospective, we don't want our users to touch tarball settings because it is hard to control/monitor these settings by Ambari or other tools. Otherwise, we have to dig into tarball settings when weird things happens. In this sense, I think it could be simpler if we don't differentiate client-side and server-side setting in configuration loading. bq. The risk of this change is when the client's *-site.xml files do not match what should be there. For the "server side" settings this has been working because we've been ignoring job.xml for those. Once we start using job.xml for even those properties, jobs that were working in the past because we ignored bad values will break. I don't know offhand how many other properties besides this one could suddenly change because we start using the client's version of the property in job.xml when we didn't before. I think even before, our configuration loading doesn't guarantee we only loading client-side configuration rather than server-side configuration. Even for mapred-site.xml, there are history server related settings that belongs to server-side but you can override a different value in client side which just doesn't work though. So we still need to rely on code logic to bypass what settings belongs to server-setting in client side. Also, better documentation could be helpful for user to differentiate what's client-side and what's server-side. Our current tricky loading mechanism won't help on both side. > Configuration to use host name in delegation token service is not read from > job.xml during MapReduce job execution. > ------------------------------------------------------------------------------------------------------------------- > > Key: MAPREDUCE-6565 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6565 > Project: Hadoop Map/Reduce > Issue Type: Bug > Reporter: Chris Nauroth > Assignee: Li Lu > > By default, the service field of a delegation token is populated based on > server IP address. Setting {{hadoop.security.token.service.use_ip}} to > {{false}} changes this behavior to use host name instead of IP address. > However, this configuration property is not read from job.xml. Instead, it's > read from a separate {{Configuration}} instance created during static > initialization of {{SecurityUtil}}. This does not work correctly with > MapReduce jobs if the framework is distributed by setting > {{mapreduce.application.framework.path}} and the > {{mapreduce.application.classpath}} is isolated to avoid reading > core-site.xml from the cluster nodes. MapReduce tasks will fail to > authenticate to HDFS, because they'll try to find a delegation token based on > the NameNode IP address, even though at job submission time the tokens were > generated using the host name. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org