[jira] [Commented] (YARN-1866) YARN RM fails to load state store with delegation token parsing error
[ https://issues.apache.org/jira/browse/YARN-1866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13947812#comment-13947812 ] Hudson commented on YARN-1866: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #521 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/521/]) YARN-1866. Fixed an issue with renewal of RM-delegation tokens on restart or fail-over. Contributed by Jian He. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1581448) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/security/DelegationTokenRenewer.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMRestart.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestDelegationTokenRenewer.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestDelegationTokenRenewerLifecycle.java YARN RM fails to load state store with delegation token parsing error - Key: YARN-1866 URL: https://issues.apache.org/jira/browse/YARN-1866 Project: Hadoop YARN Issue Type: Bug Affects Versions: 2.4.0 Reporter: Arpit Gupta Assignee: Jian He Priority: Blocker Fix For: 2.4.0 Attachments: YARN-1866.1.patch, YARN-1866.2.patch In our secure Nightlies we saw exceptions in the RM log where it failed to parse the deletegation token. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-1866) YARN RM fails to load state store with delegation token parsing error
[ https://issues.apache.org/jira/browse/YARN-1866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13947926#comment-13947926 ] Hudson commented on YARN-1866: -- SUCCESS: Integrated in Hadoop-Hdfs-trunk #1713 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1713/]) YARN-1866. Fixed an issue with renewal of RM-delegation tokens on restart or fail-over. Contributed by Jian He. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1581448) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/security/DelegationTokenRenewer.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMRestart.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestDelegationTokenRenewer.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestDelegationTokenRenewerLifecycle.java YARN RM fails to load state store with delegation token parsing error - Key: YARN-1866 URL: https://issues.apache.org/jira/browse/YARN-1866 Project: Hadoop YARN Issue Type: Bug Affects Versions: 2.4.0 Reporter: Arpit Gupta Assignee: Jian He Priority: Blocker Fix For: 2.4.0 Attachments: YARN-1866.1.patch, YARN-1866.2.patch In our secure Nightlies we saw exceptions in the RM log where it failed to parse the deletegation token. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-1866) YARN RM fails to load state store with delegation token parsing error
[ https://issues.apache.org/jira/browse/YARN-1866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13947971#comment-13947971 ] Hudson commented on YARN-1866: -- SUCCESS: Integrated in Hadoop-Mapreduce-trunk #1738 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1738/]) YARN-1866. Fixed an issue with renewal of RM-delegation tokens on restart or fail-over. Contributed by Jian He. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1581448) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/security/DelegationTokenRenewer.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMRestart.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestDelegationTokenRenewer.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestDelegationTokenRenewerLifecycle.java YARN RM fails to load state store with delegation token parsing error - Key: YARN-1866 URL: https://issues.apache.org/jira/browse/YARN-1866 Project: Hadoop YARN Issue Type: Bug Affects Versions: 2.4.0 Reporter: Arpit Gupta Assignee: Jian He Priority: Blocker Fix For: 2.4.0 Attachments: YARN-1866.1.patch, YARN-1866.2.patch In our secure Nightlies we saw exceptions in the RM log where it failed to parse the deletegation token. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-1866) YARN RM fails to load state store with delegation token parsing error
[ https://issues.apache.org/jira/browse/YARN-1866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13946994#comment-13946994 ] Hudson commented on YARN-1866: -- SUCCESS: Integrated in Hadoop-trunk-Commit #5400 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/5400/]) YARN-1866. Fixed an issue with renewal of RM-delegation tokens on restart or fail-over. Contributed by Jian He. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1581448) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/security/DelegationTokenRenewer.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMRestart.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestDelegationTokenRenewer.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestDelegationTokenRenewerLifecycle.java YARN RM fails to load state store with delegation token parsing error - Key: YARN-1866 URL: https://issues.apache.org/jira/browse/YARN-1866 Project: Hadoop YARN Issue Type: Bug Affects Versions: 2.4.0 Reporter: Arpit Gupta Assignee: Jian He Priority: Blocker Fix For: 2.4.0 Attachments: YARN-1866.1.patch, YARN-1866.2.patch In our secure Nightlies we saw exceptions in the RM log where it failed to parse the deletegation token. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-1866) YARN RM fails to load state store with delegation token parsing error
[
https://issues.apache.org/jira/browse/YARN-1866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13945613#comment-13945613
]
Arpit Gupta commented on YARN-1866:
---
{code}
interface org.apache.hadoop.ha.HAServiceProtocol
2014-03-23 17:14:49,300 INFO client.ConfiguredRMFailoverProxyProvider
(ConfiguredRMFailoverProxyProvider.java:performFailover(100)) - Failing over to
rm1
2014-03-23 17:14:49,303 WARN retry.RetryInvocationHandler
(RetryInvocationHandler.java:invoke(119)) - Exception while invoking class
org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.renewDelegationToken
over rm1. Not retrying because failovers (30) exceeded maximum allowed (30)
java.net.ConnectException: Call From host/ip to host:8032 failed on connection
exception: java.net.ConnectException: Connection refused; For more details see:
http://wiki.apache.org/hadoop/ConnectionRefused
at sun.reflect.GeneratedConstructorAccessor19.newInstance(Unknown
Source)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:783)
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:730)
at org.apache.hadoop.ipc.Client.call(Client.java:1414)
at org.apache.hadoop.ipc.Client.call(Client.java:1363)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
at $Proxy11.renewDelegationToken(Unknown Source)
at
org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.renewDelegationToken(ApplicationClientProtocolPBClientImpl.java:297)
at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:190)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:103)
at $Proxy12.renewDelegationToken(Unknown Source)
at
org.apache.hadoop.yarn.security.client.RMDelegationTokenIdentifier$Renewer.renew(RMDelegationTokenIdentifier.java:107)
at org.apache.hadoop.security.token.Token.renew(Token.java:377)
at
org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer$1.run(DelegationTokenRenewer.java:466)
at
org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer$1.run(DelegationTokenRenewer.java:463)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
at
org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer.renewToken(DelegationTokenRenewer.java:462)
at
org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer.handleAppSubmitEvent(DelegationTokenRenewer.java:384)
at
org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer.addApplicationSync(DelegationTokenRenewer.java:346)
at
org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.recoverApplication(RMAppManager.java:326)
at
org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.recover(RMAppManager.java:425)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.recover(ResourceManager.java:1018)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceStart(ResourceManager.java:481)
at
org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startActiveServices(ResourceManager.java:830)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:870)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:867)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.transitionToActive(ResourceManager.java:867)
at
org.apache.hadoop.yarn.server.resourcemanager.AdminService.transitionToActive(AdminService.java:265)
at
org.apache.hadoop.yarn.server.resourcemanager.EmbeddedElectorService.becomeActive(EmbeddedElectorService.java:116)
at
[jira] [Commented] (YARN-1866) YARN RM fails to load state store with delegation token parsing error
[ https://issues.apache.org/jira/browse/YARN-1866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13945895#comment-13945895 ] Jian He commented on YARN-1866: --- This is due to RMDelegationTokenIdentifier#localServiceAddress is not yet set when application is renewing the local tokens on recovery. YARN-1107 had a similar issue, where the test case didn't catch this because RMDelegationTokenIdentifier#localServiceAddress is a static field and once it's set in the first RM, the second RM can also pick up the same value. Upload a patch to set the localSecretManager and service address in DelegationTokenRenwer#serviceInit also. YARN RM fails to load state store with delegation token parsing error - Key: YARN-1866 URL: https://issues.apache.org/jira/browse/YARN-1866 Project: Hadoop YARN Issue Type: Bug Affects Versions: 2.4.0 Reporter: Arpit Gupta Assignee: Jian He Priority: Blocker In our secure Nightlies we saw exceptions in the RM log where it failed to parse the deletegation token. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-1866) YARN RM fails to load state store with delegation token parsing error
[ https://issues.apache.org/jira/browse/YARN-1866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13946028#comment-13946028 ] Vinod Kumar Vavilapalli commented on YARN-1866: --- The changes look good to me. +1. This part of the code keeps breaking despite all the tests. Here's hoping this is the last of the issues. Will check this in if Jenkins says okay.. YARN RM fails to load state store with delegation token parsing error - Key: YARN-1866 URL: https://issues.apache.org/jira/browse/YARN-1866 Project: Hadoop YARN Issue Type: Bug Affects Versions: 2.4.0 Reporter: Arpit Gupta Assignee: Jian He Priority: Blocker Attachments: YARN-1866.1.patch In our secure Nightlies we saw exceptions in the RM log where it failed to parse the deletegation token. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-1866) YARN RM fails to load state store with delegation token parsing error
[
https://issues.apache.org/jira/browse/YARN-1866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13946046#comment-13946046
]
Hadoop QA commented on YARN-1866:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12636470/YARN-1866.1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
org.apache.hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer
org.apache.hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewerLifecycle
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/3449//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3449//console
This message is automatically generated.
YARN RM fails to load state store with delegation token parsing error
-
Key: YARN-1866
URL: https://issues.apache.org/jira/browse/YARN-1866
Project: Hadoop YARN
Issue Type: Bug
Affects Versions: 2.4.0
Reporter: Arpit Gupta
Assignee: Jian He
Priority: Blocker
Attachments: YARN-1866.1.patch
In our secure Nightlies we saw exceptions in the RM log where it failed to
parse the deletegation token.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-1866) YARN RM fails to load state store with delegation token parsing error
[ https://issues.apache.org/jira/browse/YARN-1866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13946092#comment-13946092 ] Jian He commented on YARN-1866: --- Fixed the test failures. The test failures were due to in the tests, the rmContext is a mock object. rmContext#getClientRMService()#getBindAddress() throws NPE. YARN RM fails to load state store with delegation token parsing error - Key: YARN-1866 URL: https://issues.apache.org/jira/browse/YARN-1866 Project: Hadoop YARN Issue Type: Bug Affects Versions: 2.4.0 Reporter: Arpit Gupta Assignee: Jian He Priority: Blocker Attachments: YARN-1866.1.patch, YARN-1866.2.patch In our secure Nightlies we saw exceptions in the RM log where it failed to parse the deletegation token. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-1866) YARN RM fails to load state store with delegation token parsing error
[
https://issues.apache.org/jira/browse/YARN-1866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13946141#comment-13946141
]
Hadoop QA commented on YARN-1866:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12636510/YARN-1866.2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/3451//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3451//console
This message is automatically generated.
YARN RM fails to load state store with delegation token parsing error
-
Key: YARN-1866
URL: https://issues.apache.org/jira/browse/YARN-1866
Project: Hadoop YARN
Issue Type: Bug
Affects Versions: 2.4.0
Reporter: Arpit Gupta
Assignee: Jian He
Priority: Blocker
Attachments: YARN-1866.1.patch, YARN-1866.2.patch
In our secure Nightlies we saw exceptions in the RM log where it failed to
parse the deletegation token.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
