[
https://issues.apache.org/jira/browse/MAPREDUCE-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13129741#comment-13129741
]
Thomas Graves commented on MAPREDUCE-3181:
------------------------------------------
I haven't looked in detail at this but taking a quick look the bug doesn't show
up without kerberos enabled because it doesn't ever try to load the
yarn-site.xml without security on. Looking at the stack trace:
at
org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:83)
at
org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:205)
The obtainTokensForNamenodes eventually calls Master.getMasterPrincipal(conf)
but only if security is enabled. There is if isSecurityEnabled in
obtainTokensForNamenodes. getMasterPrinicipal then expects yarn conf to be
loaded because if the framework is yarn it does:
masterAddress = conf.get(YarnConfiguration.RM_ADDRESS,
YarnConfiguration.DEFAULT_RM_ADDRESS);
Its not all jobs that fail. teragen, wordcount, etc. work fine but they don't
take the same path.
> Terasort fails with Kerberos exception on secure cluster
> --------------------------------------------------------
>
> Key: MAPREDUCE-3181
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3181
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Anupam Seth
> Assignee: Anupam Seth
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3181-branch0_23.patch,
> MAPREDUCE-3181-branch0_23.patch
>
>
> We are seeing the following Kerberos exception upon trying to run terasort on
> secure single and multi-node clusters using the latest build from branch 0.23.
> java.io.IOException: Can't get JobTracker Kerberos principal for use as
> renewer
> at
> org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:106)
> at
> org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:90)
> at
> org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:83)
> at
> org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:205)
> at
> org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:269)
> at
> org.apache.hadoop.examples.terasort.TeraInputFormat.getSplits(TeraInputFormat.java:318)
> at
> org.apache.hadoop.examples.terasort.TeraInputFormat.writePartitionFile(TeraInputFormat.java:169)
> at org.apache.hadoop.examples.terasort.TeraSort.run(TeraSort.java:306)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
> at
> org.apache.hadoop.examples.terasort.TeraSort.main(TeraSort.java:325)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
> at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:144)
> at
> org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:68)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:189)
> Adding debug output shows that the job configuration is not loading up
> yarn-site.xml causing the above failure to happen.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
