[jira] [Commented] (MAPREDUCE-3940) ContainerTokens should have an expiry interval
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13411524#comment-13411524 ] Hudson commented on MAPREDUCE-3940: --- Integrated in Hadoop-Mapreduce-trunk #1133 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1133/]) MAPREDUCE-3940. ContainerTokens should have an expiry interval. Contributed by Siddharth Seth and Vinod Kumar Vavilapalli. (Revision 1359910) Result = SUCCESS vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359910 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRMContainerAllocator.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ContainerToken.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/ContainerManagerPBClientImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/ContainerTokenIdentifier.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/security/ContainerTokenSecretManager.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/ContainerManagerImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestEventFlow.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/BaseContainerManagerTest.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/TestContainerManager.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmcontainer/ContainerAllocationExpirer.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/resourcetracker/TestNMExpiry.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/resourcetracker/TestRMNMRPCResponseId.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/TestContainerManagerSecurity.java > ContainerTokens should have an expiry interval > -- > > Key: MAPREDUCE-3940 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2, security >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Fix For: 0.23.3 > > Attachments: MAPREDUCE-3940-20120308.txt, > MAPREDUCE-3940-20120416.txt, MAPREDUCE-3940-20120425.txt, > MAPREDUCE-3940-20120709.txt, MAPREDUCE-3940-20120710.txt, MR3940.txt, > MR3940.txt > > > - RM should generate the expiry time for a container > - A ContainerToken should have its expire time encoded > - NMs should reject containers with expired tokens. > - Expiry interval for a ContainerToken is same as the expiry interval for a > container. -- This message is automatically generated by JIRA. If yo
[jira] [Commented] (MAPREDUCE-3940) ContainerTokens should have an expiry interval
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13411503#comment-13411503 ] Daryn Sharp commented on MAPREDUCE-3940: True, the expiry check can't be universal for all operations. An expiry check in the secret manager wouldn't necessary have to be automatically invoked during token validation at the RPC level. It could just be a method that is called when validating the token for a start container. It's not a big deal, but having the expiry validation outside the secret manager seems a bit odd since the expiry is an implementation detail of the secret manager. After more throughly reviewing the code, it doesn't appear the tokens expire after the container completes? More importantly, the token doesn't appear to become invalid for launching a container after the token has already been used to launch a container? > ContainerTokens should have an expiry interval > -- > > Key: MAPREDUCE-3940 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2, security >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Fix For: 0.23.3 > > Attachments: MAPREDUCE-3940-20120308.txt, > MAPREDUCE-3940-20120416.txt, MAPREDUCE-3940-20120425.txt, > MAPREDUCE-3940-20120709.txt, MAPREDUCE-3940-20120710.txt, MR3940.txt, > MR3940.txt > > > - RM should generate the expiry time for a container > - A ContainerToken should have its expire time encoded > - NMs should reject containers with expired tokens. > - Expiry interval for a ContainerToken is same as the expiry interval for a > container. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3940) ContainerTokens should have an expiry interval
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13411478#comment-13411478 ] Hudson commented on MAPREDUCE-3940: --- Integrated in Hadoop-Hdfs-0.23-Build #310 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/310/]) MAPREDUCE-3940. ContainerTokens should have an expiry interval. Contributed by Siddharth Seth and Vinod Kumar Vavilapalli. svn merge -c 1359910 --ignore-ancestry ../../trunk/ (Revision 1359913) Result = UNSTABLE vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359913 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRMContainerAllocator.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ContainerToken.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/ContainerManagerPBClientImpl.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/ContainerTokenIdentifier.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/security/ContainerTokenSecretManager.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/ContainerManagerImpl.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestEventFlow.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/BaseContainerManagerTest.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/TestContainerManager.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmcontainer/ContainerAllocationExpirer.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/resourcetracker/TestNMExpiry.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/resourcetracker/TestRMNMRPCResponseId.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/TestContainerManagerSecurity.java > ContainerTokens should have an expiry interval > -- > > Key: MAPREDUCE-3940 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2, security >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Fix For: 0.23.3 > > Attachments: MAPREDUCE-3940-20120308.txt, > MAPREDUCE-3940-20120416.txt, MAPREDUCE-3940-20120425.txt, > MAPREDUCE-3940-20120709.txt, MAPREDUCE-3940-20120710.txt, MR3940.txt, > MR3940.txt > >
[jira] [Commented] (MAPREDUCE-3940) ContainerTokens should have an expiry interval
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13411410#comment-13411410 ] Hudson commented on MAPREDUCE-3940: --- Integrated in Hadoop-Hdfs-trunk #1100 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1100/]) MAPREDUCE-3940. ContainerTokens should have an expiry interval. Contributed by Siddharth Seth and Vinod Kumar Vavilapalli. (Revision 1359910) Result = FAILURE vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359910 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRMContainerAllocator.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ContainerToken.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/ContainerManagerPBClientImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/ContainerTokenIdentifier.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/security/ContainerTokenSecretManager.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/ContainerManagerImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestEventFlow.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/BaseContainerManagerTest.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/TestContainerManager.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmcontainer/ContainerAllocationExpirer.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/resourcetracker/TestNMExpiry.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/resourcetracker/TestRMNMRPCResponseId.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/TestContainerManagerSecurity.java > ContainerTokens should have an expiry interval > -- > > Key: MAPREDUCE-3940 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2, security >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Fix For: 0.23.3 > > Attachments: MAPREDUCE-3940-20120308.txt, > MAPREDUCE-3940-20120416.txt, MAPREDUCE-3940-20120425.txt, > MAPREDUCE-3940-20120709.txt, MAPREDUCE-3940-20120710.txt, MR3940.txt, > MR3940.txt > > > - RM should generate the expiry time for a container > - A ContainerToken should have its expire time encoded > - NMs should reject containers with expired tokens. > - Expiry interval for a ContainerToken is same as the expiry interval for a > container. -- This message is automatically generated by JIRA. If you think it
[jira] [Commented] (MAPREDUCE-3940) ContainerTokens should have an expiry interval
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13411036#comment-13411036 ] Hudson commented on MAPREDUCE-3940: --- Integrated in Hadoop-Hdfs-trunk-Commit #2511 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2511/]) MAPREDUCE-3940. ContainerTokens should have an expiry interval. Contributed by Siddharth Seth and Vinod Kumar Vavilapalli. (Revision 1359910) Result = SUCCESS vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359910 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRMContainerAllocator.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ContainerToken.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/ContainerManagerPBClientImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/ContainerTokenIdentifier.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/security/ContainerTokenSecretManager.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/ContainerManagerImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestEventFlow.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/BaseContainerManagerTest.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/TestContainerManager.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmcontainer/ContainerAllocationExpirer.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/resourcetracker/TestNMExpiry.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/resourcetracker/TestRMNMRPCResponseId.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/TestContainerManagerSecurity.java > ContainerTokens should have an expiry interval > -- > > Key: MAPREDUCE-3940 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2, security >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Fix For: 0.23.3 > > Attachments: MAPREDUCE-3940-20120308.txt, > MAPREDUCE-3940-20120416.txt, MAPREDUCE-3940-20120425.txt, > MAPREDUCE-3940-20120709.txt, MAPREDUCE-3940-20120710.txt, MR3940.txt, > MR3940.txt > > > - RM should generate the expiry time for a container > - A ContainerToken should have its expire time encoded > - NMs should reject containers with expired tokens. > - Expiry interval for a ContainerToken is same as the expiry interval for a > container. -- This message is automatically generated by JIRA. I
[jira] [Commented] (MAPREDUCE-3940) ContainerTokens should have an expiry interval
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13411030#comment-13411030 ] Hudson commented on MAPREDUCE-3940: --- Integrated in Hadoop-Common-trunk-Commit #2444 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2444/]) MAPREDUCE-3940. ContainerTokens should have an expiry interval. Contributed by Siddharth Seth and Vinod Kumar Vavilapalli. (Revision 1359910) Result = SUCCESS vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359910 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRMContainerAllocator.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ContainerToken.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/ContainerManagerPBClientImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/ContainerTokenIdentifier.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/security/ContainerTokenSecretManager.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/ContainerManagerImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestEventFlow.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/BaseContainerManagerTest.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/TestContainerManager.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmcontainer/ContainerAllocationExpirer.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/resourcetracker/TestNMExpiry.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/resourcetracker/TestRMNMRPCResponseId.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/TestContainerManagerSecurity.java > ContainerTokens should have an expiry interval > -- > > Key: MAPREDUCE-3940 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2, security >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Fix For: 0.23.3 > > Attachments: MAPREDUCE-3940-20120308.txt, > MAPREDUCE-3940-20120416.txt, MAPREDUCE-3940-20120425.txt, > MAPREDUCE-3940-20120709.txt, MAPREDUCE-3940-20120710.txt, MR3940.txt, > MR3940.txt > > > - RM should generate the expiry time for a container > - A ContainerToken should have its expire time encoded > - NMs should reject containers with expired tokens. > - Expiry interval for a ContainerToken is same as the expiry interval for a > container. -- This message is automatically generated by JIR
[jira] [Commented] (MAPREDUCE-3940) ContainerTokens should have an expiry interval
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13411028#comment-13411028 ] Hudson commented on MAPREDUCE-3940: --- Integrated in Hadoop-Mapreduce-trunk-Commit #2462 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2462/]) MAPREDUCE-3940. ContainerTokens should have an expiry interval. Contributed by Siddharth Seth and Vinod Kumar Vavilapalli. (Revision 1359910) Result = FAILURE vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359910 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRMContainerAllocator.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ContainerToken.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/ContainerManagerPBClientImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/ContainerTokenIdentifier.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/security/ContainerTokenSecretManager.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/ContainerManagerImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestEventFlow.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/BaseContainerManagerTest.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/TestContainerManager.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmcontainer/ContainerAllocationExpirer.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/resourcetracker/TestNMExpiry.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/resourcetracker/TestRMNMRPCResponseId.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/TestContainerManagerSecurity.java > ContainerTokens should have an expiry interval > -- > > Key: MAPREDUCE-3940 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2, security >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Fix For: 0.23.3 > > Attachments: MAPREDUCE-3940-20120308.txt, > MAPREDUCE-3940-20120416.txt, MAPREDUCE-3940-20120425.txt, > MAPREDUCE-3940-20120709.txt, MAPREDUCE-3940-20120710.txt, MR3940.txt, > MR3940.txt > > > - RM should generate the expiry time for a container > - A ContainerToken should have its expire time encoded > - NMs should reject containers with expired tokens. > - Expiry interval for a ContainerToken is same as the expiry interval for a > container. -- This message is automatically generated
[jira] [Commented] (MAPREDUCE-3940) ContainerTokens should have an expiry interval
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13410911#comment-13410911 ] Vinod Kumar Vavilapalli commented on MAPREDUCE-3940: bq. For consistency with other managers, would it make more sense for the expiration check to be in the secret manager, and for it to throw an InvalidToken exception? Unfortunately that can't be. That was how the earlier versions of the patch were implemented but as Sid rightly caught a (big) bug above, that would make us use the same expiry checks for stopContainer() and getContainerStatus() calls as well, which is wrong - essentially nobody would be able to stop containers or get statuses. > ContainerTokens should have an expiry interval > -- > > Key: MAPREDUCE-3940 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2, security >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Attachments: MAPREDUCE-3940-20120308.txt, > MAPREDUCE-3940-20120416.txt, MAPREDUCE-3940-20120425.txt, > MAPREDUCE-3940-20120709.txt, MAPREDUCE-3940-20120710.txt, MR3940.txt, > MR3940.txt > > > - RM should generate the expiry time for a container > - A ContainerToken should have its expire time encoded > - NMs should reject containers with expired tokens. > - Expiry interval for a ContainerToken is same as the expiry interval for a > container. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3940) ContainerTokens should have an expiry interval
[
https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13410891#comment-13410891
]
Daryn Sharp commented on MAPREDUCE-3940:
Haven't fully reviewed it, but what I'm seeing is looking good. For
consistency with other managers, would it make more sense for the expiration
check to be in the secret manager, and for it to throw an {{InvalidToken}}
exception?
> ContainerTokens should have an expiry interval
> --
>
> Key: MAPREDUCE-3940
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
>Affects Versions: 0.23.0
>Reporter: Vinod Kumar Vavilapalli
>Assignee: Vinod Kumar Vavilapalli
> Attachments: MAPREDUCE-3940-20120308.txt,
> MAPREDUCE-3940-20120416.txt, MAPREDUCE-3940-20120425.txt,
> MAPREDUCE-3940-20120709.txt, MAPREDUCE-3940-20120710.txt, MR3940.txt,
> MR3940.txt
>
>
> - RM should generate the expiry time for a container
> - A ContainerToken should have its expire time encoded
> - NMs should reject containers with expired tokens.
> - Expiry interval for a ContainerToken is same as the expiry interval for a
> container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3940) ContainerTokens should have an expiry interval
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13410822#comment-13410822 ] Siddharth Seth commented on MAPREDUCE-3940: --- +1. This looks good. Thanks Vinod > ContainerTokens should have an expiry interval > -- > > Key: MAPREDUCE-3940 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2, security >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Attachments: MAPREDUCE-3940-20120308.txt, > MAPREDUCE-3940-20120416.txt, MAPREDUCE-3940-20120425.txt, > MAPREDUCE-3940-20120709.txt, MAPREDUCE-3940-20120710.txt, MR3940.txt, > MR3940.txt > > > - RM should generate the expiry time for a container > - A ContainerToken should have its expire time encoded > - NMs should reject containers with expired tokens. > - Expiry interval for a ContainerToken is same as the expiry interval for a > container. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3940) ContainerTokens should have an expiry interval
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13410753#comment-13410753 ] Hadoop QA commented on MAPREDUCE-3940: -- +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12535903/MAPREDUCE-3940-20120710.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 7 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2567//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2567//console This message is automatically generated. > ContainerTokens should have an expiry interval > -- > > Key: MAPREDUCE-3940 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2, security >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Attachments: MAPREDUCE-3940-20120308.txt, > MAPREDUCE-3940-20120416.txt, MAPREDUCE-3940-20120425.txt, > MAPREDUCE-3940-20120709.txt, MAPREDUCE-3940-20120710.txt, MR3940.txt, > MR3940.txt > > > - RM should generate the expiry time for a container > - A ContainerToken should have its expire time encoded > - NMs should reject containers with expired tokens. > - Expiry interval for a ContainerToken is same as the expiry interval for a > container. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3940) ContainerTokens should have an expiry interval
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13410706#comment-13410706 ] Siddharth Seth commented on MAPREDUCE-3940: --- Needs one change in the FifoScheduler. By returning early on a token creation failure - accounting of 'usedResource' can go wrong. Otherwise looks good. > ContainerTokens should have an expiry interval > -- > > Key: MAPREDUCE-3940 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2, security >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Attachments: MAPREDUCE-3940-20120308.txt, > MAPREDUCE-3940-20120416.txt, MAPREDUCE-3940-20120425.txt, > MAPREDUCE-3940-20120709.txt, MR3940.txt, MR3940.txt > > > - RM should generate the expiry time for a container > - A ContainerToken should have its expire time encoded > - NMs should reject containers with expired tokens. > - Expiry interval for a ContainerToken is same as the expiry interval for a > container. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3940) ContainerTokens should have an expiry interval
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13410183#comment-13410183 ] Hadoop QA commented on MAPREDUCE-3940: -- +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12535744/MAPREDUCE-3940-20120709.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 7 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2563//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2563//console This message is automatically generated. > ContainerTokens should have an expiry interval > -- > > Key: MAPREDUCE-3940 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2, security >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Attachments: MAPREDUCE-3940-20120308.txt, > MAPREDUCE-3940-20120416.txt, MAPREDUCE-3940-20120425.txt, > MAPREDUCE-3940-20120709.txt, MR3940.txt, MR3940.txt > > > - RM should generate the expiry time for a container > - A ContainerToken should have its expire time encoded > - NMs should reject containers with expired tokens. > - Expiry interval for a ContainerToken is same as the expiry interval for a > container. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3940) ContainerTokens should have an expiry interval
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13288952#comment-13288952 ] Siddharth Seth commented on MAPREDUCE-3940: --- bq. The ContainerTokenSecretManager appears to be using a hardcoded secret of mySecretKey? That's fixed in MAPREDUCE-3943. bq. Secret managers usually handle the expiration internally to prevent tampering, but the token ident includes the expiry. Combined with the prior point, is it possible to fabricate tokens for any host with any expiration? Given the hardcoded secret, that's possible. Again, 3943 fixes this. bq. The secret manager usually validates the token & expiration, but here it appears the container manager itself is trying to do it? Does this mean there's no SASL level token check occurring? The secret manager continues to validate parts of token. Expiry is done outside since that's only required for the startContainer call. bq. The UGI is the container id instead of the job's submitter? That's used to ensure that the RM did allocate the specific container. bq. The schedulers (fifo & leaf) and secret manager interaction seem inconsistent with the other implementations. Token(ident, secretManager) seems to be the preferred way to create tokens. The token identifier and signature are generated by the RM - serialized using PB and then uesd by the AM to communicate with the node manager. The AM generates the Token from the PB message. The RM doesn't really need to create it. > ContainerTokens should have an expiry interval > -- > > Key: MAPREDUCE-3940 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2, security >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Attachments: MAPREDUCE-3940-20120308.txt, > MAPREDUCE-3940-20120416.txt, MAPREDUCE-3940-20120425.txt, MR3940.txt, > MR3940.txt > > > - RM should generate the expiry time for a container > - A ContainerToken should have its expire time encoded > - NMs should reject containers with expired tokens. > - Expiry interval for a ContainerToken is same as the expiry interval for a > container. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3940) ContainerTokens should have an expiry interval
[
https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13286752#comment-13286752
]
Daryn Sharp commented on MAPREDUCE-3940:
A few questions/concerns:
# The {{ContainerTokenSecretManager}} appears to be using a hardcoded secret of
{{mySecretKey}}?
# Secret managers usually handle the expiration internally to prevent
tampering, but the token ident includes the expiry. Combined with the prior
point, is it possible to fabricate tokens for any host with any expiration?
# The secret manager usually validates the token & expiration, but here it
appears the container manager itself is trying to do it? Does this mean
there's no SASL level token check occurring?
# The UGI is the container id instead of the job's submitter?
# The schedulers (fifo & leaf) and secret manager interaction seem inconsistent
with the other implementations. {{Token(ident, secretManager)}} seems to be
the preferred way to create tokens.
> ContainerTokens should have an expiry interval
> --
>
> Key: MAPREDUCE-3940
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2, security
>Affects Versions: 0.23.0
>Reporter: Vinod Kumar Vavilapalli
>Assignee: Vinod Kumar Vavilapalli
> Attachments: MAPREDUCE-3940-20120308.txt,
> MAPREDUCE-3940-20120416.txt, MAPREDUCE-3940-20120425.txt, MR3940.txt,
> MR3940.txt
>
>
> - RM should generate the expiry time for a container
> - A ContainerToken should have its expire time encoded
> - NMs should reject containers with expired tokens.
> - Expiry interval for a ContainerToken is same as the expiry interval for a
> container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3940) ContainerTokens should have an expiry interval
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13274775#comment-13274775 ] Hadoop QA commented on MAPREDUCE-3940: -- +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12526779/MR3940.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 7 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2387//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2387//console This message is automatically generated. > ContainerTokens should have an expiry interval > -- > > Key: MAPREDUCE-3940 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2, security >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Attachments: MAPREDUCE-3940-20120308.txt, > MAPREDUCE-3940-20120416.txt, MAPREDUCE-3940-20120425.txt, MR3940.txt, > MR3940.txt > > > - RM should generate the expiry time for a container > - A ContainerToken should have its expire time encoded > - NMs should reject containers with expired tokens. > - Expiry interval for a ContainerToken is same as the expiry interval for a > container. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3940) ContainerTokens should have an expiry interval
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13264141#comment-13264141 ] Hadoop QA commented on MAPREDUCE-3940: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12524945/MR3940.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 7 new or modified test files. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.mapreduce.TestYarnClientProtocolProvider +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2327//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2327//console This message is automatically generated. > ContainerTokens should have an expiry interval > -- > > Key: MAPREDUCE-3940 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2, security >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Attachments: MAPREDUCE-3940-20120308.txt, > MAPREDUCE-3940-20120416.txt, MAPREDUCE-3940-20120425.txt, MR3940.txt > > > - RM should generate the expiry time for a container > - A ContainerToken should have its expire time encoded > - NMs should reject containers with expired tokens. > - Expiry interval for a ContainerToken is same as the expiry interval for a > container. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3940) ContainerTokens should have an expiry interval
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13261977#comment-13261977 ] Hadoop QA commented on MAPREDUCE-3940: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12524315/MAPREDUCE-3940-20120425.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 7 new or modified test files. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.yarn.server.TestContainerManagerSecurity org.apache.hadoop.yarn.server.resourcemanager.security.TestApplicationTokens org.apache.hadoop.yarn.server.resourcemanager.TestClientRMService org.apache.hadoop.yarn.server.resourcemanager.resourcetracker.TestNMExpiry org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization org.apache.hadoop.yarn.server.resourcemanager.TestApplicationACLs org.apache.hadoop.mapred.TestClientRedirect org.apache.hadoop.mapreduce.TestYarnClientProtocolProvider org.apache.hadoop.mapreduce.security.TestJHSSecurity +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2307//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2307//console This message is automatically generated. > ContainerTokens should have an expiry interval > -- > > Key: MAPREDUCE-3940 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2, security >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Fix For: 0.23.2 > > Attachments: MAPREDUCE-3940-20120308.txt, > MAPREDUCE-3940-20120416.txt, MAPREDUCE-3940-20120425.txt > > > - RM should generate the expiry time for a container > - A ContainerToken should have its expire time encoded > - NMs should reject containers with expired tokens. > - Expiry interval for a ContainerToken is same as the expiry interval for a > container. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3940) ContainerTokens should have an expiry interval
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13257097#comment-13257097 ] Siddharth Seth commented on MAPREDUCE-3940: --- One issue with the patch... The MR AM will end up seing some harmless exceptions when invoking stopContainer (similarly for getContainerStatus invocations) for any container which runs beyond the configured limit (default 10 minutes). These containers will however eventually end up being killed via the RM - once the AM releases them. > ContainerTokens should have an expiry interval > -- > > Key: MAPREDUCE-3940 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2, security >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Fix For: 0.23.2 > > Attachments: MAPREDUCE-3940-20120308.txt, MAPREDUCE-3940-20120416.txt > > > - RM should generate the expiry time for a container > - A ContainerToken should have its expire time encoded > - NMs should reject containers with expired tokens. > - Expiry interval for a ContainerToken is same as the expiry interval for a > container. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3940) ContainerTokens should have an expiry interval
[ https://issues.apache.org/jira/browse/MAPREDUCE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13225642#comment-13225642 ] Hadoop QA commented on MAPREDUCE-3940: -- +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12517621/MAPREDUCE-3940-20120308.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 21 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2020//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2020//console This message is automatically generated. > ContainerTokens should have an expiry interval > -- > > Key: MAPREDUCE-3940 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3940 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mrv2, security >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Fix For: 0.23.2 > > Attachments: MAPREDUCE-3940-20120308.txt > > > - RM should generate the expiry time for a container > - A ContainerToken should have its expire time encoded > - NMs should reject containers with expired tokens. > - Expiry interval for a ContainerToken is same as the expiry interval for a > container. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
