[
https://issues.apache.org/jira/browse/MAPREDUCE-3736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13195907#comment-13195907
]
Eli Collins commented on MAPREDUCE-3736:
----------------------------------------
Actually when fs.default.name was removed from core-site by HADOOP-6233 a
deprecation was added so it would get mapped to fs.defaultFS, so updating MR to
use fs.defaultFS should be fine, if defaultFS is not defined it should do the
lookup with fs.default.name.
However, if doing a lookup on fs.default.name (as MR does currently) and it
finds that this key has no value I think it's *supposed* to do a lookup using
the new key (fs.defaultFS), so if HADOOP-6233 was done correctly as I
understand it this change shouldn't be necessary, eg a lookup to
fs.default.name should map to a lookup to fs.defaultFS (if fs.default.name was
not defined) and we do have a value for it defined in core-default.xml. Seems
like either the configuration deprecation is broken or something else going on
here (eg in YarnConfiguration somehow fs.default.name is getting defined or
overwritten).
> Variable substitution depth too large for fs.default.name causes jobs to fail
> -----------------------------------------------------------------------------
>
> Key: MAPREDUCE-3736
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3736
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.1
> Reporter: Eli Collins
> Assignee: Ahmed Radwan
> Priority: Blocker
> Attachments: MAPREDUCE-3736.patch, MAPREDUCE-3736_rev2.patch
>
>
> I'm seeing the same failure as MAPREDUCE-3462 in downstream projects running
> against a recent build of branch-23. MR-3462 modified the tests rather than
> fixing the framework. In that jira Ravi mentioned "I'm still ignorant of the
> change which made the tests start to fail. I should probably understand
> better the reasons for that change before proposing a more generalized fix."
> Let's figure out the general fix (rather than require all projects to set
> mapreduce.job.hdfs-servers in their conf we should fix this in the
> framework). Perhaps we should not default this config to "$fs.default.name"?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
