[
https://issues.apache.org/jira/browse/MAPREDUCE-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13129745#comment-13129745
]
Anupam Seth commented on MAPREDUCE-3181:
----------------------------------------
Thanks Tom. I hope your explanation helps clear up Aaron's and Arun's
questions. As Tom pointed out, this problem is seem only on Terasort in a
secure cluster because of the conditional to not even try this if security is
disabled and because this gets hit before the ResourceManagerDelegate is ever
called or created (causing yarn-site.xml to be 'normally' loaded). This does
not happen with Teragen, for example, because TeraSort attempts to write the
input splits to HDFS before running the actual MR.
Regarding the ant tests asked for by Mahadev, I see numerous failures due to
timeout issues, most likely an artifact of MAPREDUCE-3176, as my changes are
very simple.
Also, I have patch ready with an added unit test to ensure this does not
regress - am running tests on it, will put it up shortly.
> Terasort fails with Kerberos exception on secure cluster
> --------------------------------------------------------
>
> Key: MAPREDUCE-3181
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3181
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Anupam Seth
> Assignee: Anupam Seth
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3181-branch0_23.patch,
> MAPREDUCE-3181-branch0_23.patch
>
>
> We are seeing the following Kerberos exception upon trying to run terasort on
> secure single and multi-node clusters using the latest build from branch 0.23.
> java.io.IOException: Can't get JobTracker Kerberos principal for use as
> renewer
> at
> org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:106)
> at
> org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:90)
> at
> org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:83)
> at
> org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:205)
> at
> org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:269)
> at
> org.apache.hadoop.examples.terasort.TeraInputFormat.getSplits(TeraInputFormat.java:318)
> at
> org.apache.hadoop.examples.terasort.TeraInputFormat.writePartitionFile(TeraInputFormat.java:169)
> at org.apache.hadoop.examples.terasort.TeraSort.run(TeraSort.java:306)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
> at
> org.apache.hadoop.examples.terasort.TeraSort.main(TeraSort.java:325)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
> at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:144)
> at
> org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:68)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:189)
> Adding debug output shows that the job configuration is not loading up
> yarn-site.xml causing the above failure to happen.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
