[
https://issues.apache.org/jira/browse/MAPREDUCE-5375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinod Kumar Vavilapalli updated MAPREDUCE-5375:
-----------------------------------------------
Attachment: MRDelegationIssue-MR-5375.patch
Here's a patch from [~venkatnrangan]. With this patch and with the fix for
Oozie auth_to_local issue, there are no more exceptions in the jobtracker log
complaining about delegation renewal issues.
> Delegation Token renewal exception in jobtracker logs
> -----------------------------------------------------
>
> Key: MAPREDUCE-5375
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5375
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 1.2.0
> Reporter: Venkat Ranganathan
> Assignee: Venkat Ranganathan
> Priority: Critical
> Attachments: MRDelegationIssue-MR-5375.patch
>
>
> Filing on behalf of [~venkatnrangan] who found this originally and provided a
> patch.
> Saw this in the JT logs while oozie tests were running with Hadoop.
> When Oozie java action is executed, the following shows up in the job tracker
> log.
> {code}
> ERROR org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal:
> Exception renewing tokenIdent: 00 07 68 64 70 75 73 65 72 06 6d 61 70 72 65
> 64 26 6f 6f 7a 69 65 2f 63 6f 6e 64 6f 72 2d 73 65 63 2e 76 65 6e 6b 61 74 2e
> 6f 72 67 40 76 65 6e 6b 61 74 2e 6f 72 67 8a 01 3e a6 87 5e 5b 8a 01 3e ca 93
> e2 5b 02 02, Kind: MAPREDUCE_DELEGATION_TOKEN, Service: ip:50300. Not
> rescheduled
> org.apache.hadoop.ipc.RemoteException:
> org.apache.hadoop.security.AccessControlException: Client jt/h...@domain.com
> tries to renew a token with renewer specified as mapred
> at
> org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager.renewToken(AbstractDelegationTokenSecretManager.java:267)
> at
> org.apache.hadoop.mapred.JobTracker.renewDelegationToken(JobTracker.java:3878)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:587)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1405)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1401)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1399)
> at org.apache.hadoop.ipc.Client.call(Client.java:1118)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)
> at org.apache.hadoop.mapred.$Proxy8.renewDelegationToken(Unknown
> Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)
> at org.apache.hadoop.mapred.$Proxy8.renewDelegationToken(Unknown
> Source)
> at
> org.apache.hadoop.mapred.JobClient$Renewer.renew(JobClient.java:578)
> at org.apache.hadoop.security.token.Token.renew(Token.java:309)
> at
> org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal$RenewalTimerTask$1.run(DelegationTokenRenewal.java:221)
> at
> org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal$RenewalTimerTask$1.run(DelegationTokenRenewal.java:217)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
> at
> org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal$RenewalTimerTask.run(DelegationTokenRenewal.java:216)
> at java.util.TimerThread.mainLoop(Timer.java:512)
> at java.util.TimerThread.run(Timer.java:462)
> {code}
> Setting the renewer to Kerberos Local name does not help because
> AbstractDelegationTokenIdentifier sets the renewer to Kerberos shortname but
> JobTracker.renewDelegationToken uses the fullName. This essentially causes
> the renewal to fail.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
