[
https://issues.apache.org/jira/browse/MAPREDUCE-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15674699#comment-15674699
]
Jason Lowe commented on MAPREDUCE-6565:
---------------------------------------
bq. it won't have any benefit to give client flexibility to be different with
what server settings. Isn't it?
Agreed, I think the best approach here long-term is to not have this be a
config setting at all but rather derived from the communication session that
retrieved the token in the first place. I also agree that it is going to be
more dangerous than not for job.xml to override this particular setting
(whether or not we're using an MR tarball).
bq. if we think everything inside of MR tarball should be per job only
This is definitely not the case, at least for our clusters. As I mentioned
above, the MR tarballs are created by the cluster admins and therefore have the
appropriate configs for that cluster. We do _not_ want the jobs to pick up
configs from the nodes per the issues I described earlier.
bq. I am not fully agree that it should be cluster admin' job to create tarball
and keep consistent for all configurations with cluster settings.
The tarball consist of the Hadoop jars and the *-site.xml files. Both of these
are things admins of the cluster are expected to maintain and provide.
Therefore I think it's completely appropriate that typically these tarballs are
created and provided by admins. We're not running a bunch of different
versions of the tarball for different types of jobs. We just have the main
tarball for the cluster and users override various settings for their specific
job via job.xml, just like a regular non-tarball job does.
To be clear, I'm not saying everyone has to run it the way we do. However if
you don't then there needs to be solutions for the types of rolling upgrade
problems I pointed out above. Also I don't think the way we're using the
tarball is an invalid setup, so therefore any proposed change needs to ensure
this type of setup keeps working.
> Configuration to use host name in delegation token service is not read from
> job.xml during MapReduce job execution.
> -------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-6565
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6565
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Chris Nauroth
> Assignee: Li Lu
>
> By default, the service field of a delegation token is populated based on
> server IP address. Setting {{hadoop.security.token.service.use_ip}} to
> {{false}} changes this behavior to use host name instead of IP address.
> However, this configuration property is not read from job.xml. Instead, it's
> read from a separate {{Configuration}} instance created during static
> initialization of {{SecurityUtil}}. This does not work correctly with
> MapReduce jobs if the framework is distributed by setting
> {{mapreduce.application.framework.path}} and the
> {{mapreduce.application.classpath}} is isolated to avoid reading
> core-site.xml from the cluster nodes. MapReduce tasks will fail to
> authenticate to HDFS, because they'll try to find a delegation token based on
> the NameNode IP address, even though at job submission time the tokens were
> generated using the host name.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
