Vinod Kumar Vavilapalli created MAPREDUCE-5375:
--------------------------------------------------
Summary: Delegation Token renewal exception in jobtracker logs
Key: MAPREDUCE-5375
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5375
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Venkat Ranganathan
Assignee: Venkat Ranganathan
Priority: Critical
Filing on behalf of [~venkatnrangan] who found this originally and provided a
patch.
Saw this in the JT logs while oozie tests were running with Hadoop.
When Oozie java action is executed, the following shows up in the job tracker
log.
{code}
ERROR org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal:
Exception renewing tokenIdent: 00 07 68 64 70 75 73 65 72 06 6d 61 70 72 65 64
26 6f 6f 7a 69 65 2f 63 6f 6e 64 6f 72 2d 73 65 63 2e 76 65 6e 6b 61 74 2e 6f
72 67 40 76 65 6e 6b 61 74 2e 6f 72 67 8a 01 3e a6 87 5e 5b 8a 01 3e ca 93 e2
5b 02 02, Kind: MAPREDUCE_DELEGATION_TOKEN, Service: ip:50300. Not rescheduled
org.apache.hadoop.ipc.RemoteException:
org.apache.hadoop.security.AccessControlException: Client jt/h...@domain.com
tries to renew a token with renewer specified as mapred
at
org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager.renewToken(AbstractDelegationTokenSecretManager.java:267)
at
org.apache.hadoop.mapred.JobTracker.renewDelegationToken(JobTracker.java:3878)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:587)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1405)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1401)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1399)
at org.apache.hadoop.ipc.Client.call(Client.java:1118)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)
at org.apache.hadoop.mapred.$Proxy8.renewDelegationToken(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)
at org.apache.hadoop.mapred.$Proxy8.renewDelegationToken(Unknown Source)
at org.apache.hadoop.mapred.JobClient$Renewer.renew(JobClient.java:578)
at org.apache.hadoop.security.token.Token.renew(Token.java:309)
at
org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal$RenewalTimerTask$1.run(DelegationTokenRenewal.java:221)
at
org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal$RenewalTimerTask$1.run(DelegationTokenRenewal.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
at
org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal$RenewalTimerTask.run(DelegationTokenRenewal.java:216)
at java.util.TimerThread.mainLoop(Timer.java:512)
at java.util.TimerThread.run(Timer.java:462)
{code}
Setting the renewer to Kerberos Local name does not help because
AbstractDelegationTokenIdentifier sets the renewer to Kerberos shortname but
JobTracker.renewDelegationToken uses the fullName. This essentially causes the
renewal to fail.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
