[jira] [Commented] (YARN-579) Make ApplicationToken part of Container's token list to help RM-restart
[ https://issues.apache.org/jira/browse/YARN-579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13644859#comment-13644859 ] Vinod Kumar Vavilapalli commented on YARN-579: -- I validated this on trunk, I can run it successfully on trunk even now. It seems like it is failing on branch-2. Something at RPC level I suppose, digging through.. > Make ApplicationToken part of Container's token list to help RM-restart > --- > > Key: YARN-579 > URL: https://issues.apache.org/jira/browse/YARN-579 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 2.0.4-alpha >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Fix For: 2.0.5-beta > > Attachments: YARN-579-20130422.1.txt, > YARN-579-20130422.1_YARNChanges.txt > > > Container is already persisted for helping RM restart. Instead of explicitly > setting ApplicationToken in AM's env, if we change it to be in Container, we > can avoid env and can also help restart. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-579) Make ApplicationToken part of Container's token list to help RM-restart
[ https://issues.apache.org/jira/browse/YARN-579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13641829#comment-13641829 ] Hudson commented on YARN-579: - Integrated in Hadoop-Mapreduce-trunk #1410 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1410/]) YARN-579. Stop setting the Application Token in the AppMaster env, in favour of the copy present in the container token field. Contributed by Vinod Kumar Vavilapalli. (Revision 1471814) Result = SUCCESS sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1471814 Files : * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ApplicationConstants.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/AMRMClientImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/amlauncher/AMLauncher.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestAMAuthorization.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestApplicationTokens.java > Make ApplicationToken part of Container's token list to help RM-restart > --- > > Key: YARN-579 > URL: https://issues.apache.org/jira/browse/YARN-579 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 2.0.4-alpha >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Fix For: 2.0.5-beta > > Attachments: YARN-579-20130422.1.txt, > YARN-579-20130422.1_YARNChanges.txt > > > Container is already persisted for helping RM restart. Instead of explicitly > setting ApplicationToken in AM's env, if we change it to be in Container, we > can avoid env and can also help restart. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-579) Make ApplicationToken part of Container's token list to help RM-restart
[ https://issues.apache.org/jira/browse/YARN-579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13641753#comment-13641753 ] Hudson commented on YARN-579: - Integrated in Hadoop-Hdfs-trunk #1383 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1383/]) YARN-579. Stop setting the Application Token in the AppMaster env, in favour of the copy present in the container token field. Contributed by Vinod Kumar Vavilapalli. (Revision 1471814) Result = FAILURE sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1471814 Files : * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ApplicationConstants.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/AMRMClientImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/amlauncher/AMLauncher.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestAMAuthorization.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestApplicationTokens.java > Make ApplicationToken part of Container's token list to help RM-restart > --- > > Key: YARN-579 > URL: https://issues.apache.org/jira/browse/YARN-579 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 2.0.4-alpha >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Fix For: 2.0.5-beta > > Attachments: YARN-579-20130422.1.txt, > YARN-579-20130422.1_YARNChanges.txt > > > Container is already persisted for helping RM restart. Instead of explicitly > setting ApplicationToken in AM's env, if we change it to be in Container, we > can avoid env and can also help restart. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-579) Make ApplicationToken part of Container's token list to help RM-restart
[ https://issues.apache.org/jira/browse/YARN-579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13641655#comment-13641655 ] Hudson commented on YARN-579: - Integrated in Hadoop-Yarn-trunk #194 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/194/]) YARN-579. Stop setting the Application Token in the AppMaster env, in favour of the copy present in the container token field. Contributed by Vinod Kumar Vavilapalli. (Revision 1471814) Result = SUCCESS sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1471814 Files : * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ApplicationConstants.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/AMRMClientImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/amlauncher/AMLauncher.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestAMAuthorization.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestApplicationTokens.java > Make ApplicationToken part of Container's token list to help RM-restart > --- > > Key: YARN-579 > URL: https://issues.apache.org/jira/browse/YARN-579 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 2.0.4-alpha >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Fix For: 2.0.5-beta > > Attachments: YARN-579-20130422.1.txt, > YARN-579-20130422.1_YARNChanges.txt > > > Container is already persisted for helping RM restart. Instead of explicitly > setting ApplicationToken in AM's env, if we change it to be in Container, we > can avoid env and can also help restart. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-579) Make ApplicationToken part of Container's token list to help RM-restart
[ https://issues.apache.org/jira/browse/YARN-579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13641334#comment-13641334 ] Hudson commented on YARN-579: - Integrated in Hadoop-trunk-Commit #3660 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/3660/]) YARN-579. Stop setting the Application Token in the AppMaster env, in favour of the copy present in the container token field. Contributed by Vinod Kumar Vavilapalli. (Revision 1471814) Result = SUCCESS sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1471814 Files : * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ApplicationConstants.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/AMRMClientImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/amlauncher/AMLauncher.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestAMAuthorization.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestApplicationTokens.java > Make ApplicationToken part of Container's token list to help RM-restart > --- > > Key: YARN-579 > URL: https://issues.apache.org/jira/browse/YARN-579 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 2.0.4-alpha >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Fix For: 2.0.5-beta > > Attachments: YARN-579-20130422.1.txt, > YARN-579-20130422.1_YARNChanges.txt > > > Container is already persisted for helping RM restart. Instead of explicitly > setting ApplicationToken in AM's env, if we change it to be in Container, we > can avoid env and can also help restart. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-579) Make ApplicationToken part of Container's token list to help RM-restart
[ https://issues.apache.org/jira/browse/YARN-579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13641326#comment-13641326 ] Vinod Kumar Vavilapalli commented on YARN-579: -- Jian/Bikas, shoo, get off my ticket and move onto YARN-582.. ;) > Make ApplicationToken part of Container's token list to help RM-restart > --- > > Key: YARN-579 > URL: https://issues.apache.org/jira/browse/YARN-579 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 2.0.4-alpha >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Attachments: YARN-579-20130422.1.txt, > YARN-579-20130422.1_YARNChanges.txt > > > Container is already persisted for helping RM restart. Instead of explicitly > setting ApplicationToken in AM's env, if we change it to be in Container, we > can avoid env and can also help restart. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-579) Make ApplicationToken part of Container's token list to help RM-restart
[ https://issues.apache.org/jira/browse/YARN-579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13641322#comment-13641322 ] Siddharth Seth commented on YARN-579: - bq. The AppToken is stored in ContainerLaunchContext,not in Container, so appToken should be set before ApplicationSubmissionContext is saved to help RM-restart bq. appToken is attempt specific. So it needs to be store per app attempt. So it needs to be generated when app attempt is assigned a container and saved in appAttemptStateData. Basically the code needs to be moved from AMLauncher to the rmappattempimpl. AMLauncher will simply use the token from the attempt. I believe this is being addressed in YARN-582. Committing this patch. The latest test-patch can be ignored since the MR bit is necessary for trunk to compile. > Make ApplicationToken part of Container's token list to help RM-restart > --- > > Key: YARN-579 > URL: https://issues.apache.org/jira/browse/YARN-579 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 2.0.4-alpha >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Attachments: YARN-579-20130422.1.txt, > YARN-579-20130422.1_YARNChanges.txt > > > Container is already persisted for helping RM restart. Instead of explicitly > setting ApplicationToken in AM's env, if we change it to be in Container, we > can avoid env and can also help restart. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-579) Make ApplicationToken part of Container's token list to help RM-restart
[ https://issues.apache.org/jira/browse/YARN-579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13641315#comment-13641315 ] Bikas Saha commented on YARN-579: - appToken is attempt specific. So it needs to be store per app attempt. So it needs to be generated when app attempt is assigned a container and saved in appAttemptStateData. Basically the code needs to be moved from AMLauncher to the rmappattempimpl. AMLauncher will simply use the token from the attempt. > Make ApplicationToken part of Container's token list to help RM-restart > --- > > Key: YARN-579 > URL: https://issues.apache.org/jira/browse/YARN-579 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 2.0.4-alpha >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Attachments: YARN-579-20130422.1.txt, > YARN-579-20130422.1_YARNChanges.txt > > > Container is already persisted for helping RM restart. Instead of explicitly > setting ApplicationToken in AM's env, if we change it to be in Container, we > can avoid env and can also help restart. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-579) Make ApplicationToken part of Container's token list to help RM-restart
[ https://issues.apache.org/jira/browse/YARN-579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13641273#comment-13641273 ] Jian He commented on YARN-579: -- The AppToken is stored in ContainerLaunchContext,not in Container, so appToken should be set before ApplicationSubmissionContext is saved to help RM-restart > Make ApplicationToken part of Container's token list to help RM-restart > --- > > Key: YARN-579 > URL: https://issues.apache.org/jira/browse/YARN-579 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 2.0.4-alpha >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Attachments: YARN-579-20130422.1.txt, > YARN-579-20130422.1_YARNChanges.txt > > > Container is already persisted for helping RM restart. Instead of explicitly > setting ApplicationToken in AM's env, if we change it to be in Container, we > can avoid env and can also help restart. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-579) Make ApplicationToken part of Container's token list to help RM-restart
[
https://issues.apache.org/jira/browse/YARN-579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13641264#comment-13641264
]
Hadoop QA commented on YARN-579:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12580439/YARN-579-20130422.1_YARNChanges.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:red}-1 javac{color:red}. The patch appears to cause the build to
fail.
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/819//console
This message is automatically generated.
> Make ApplicationToken part of Container's token list to help RM-restart
> ---
>
> Key: YARN-579
> URL: https://issues.apache.org/jira/browse/YARN-579
> Project: Hadoop YARN
> Issue Type: Sub-task
>Affects Versions: 2.0.4-alpha
>Reporter: Vinod Kumar Vavilapalli
>Assignee: Vinod Kumar Vavilapalli
> Attachments: YARN-579-20130422.1.txt,
> YARN-579-20130422.1_YARNChanges.txt
>
>
> Container is already persisted for helping RM restart. Instead of explicitly
> setting ApplicationToken in AM's env, if we change it to be in Container, we
> can avoid env and can also help restart.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-579) Make ApplicationToken part of Container's token list to help RM-restart
[ https://issues.apache.org/jira/browse/YARN-579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13641251#comment-13641251 ] Siddharth Seth commented on YARN-579: - +1. The patch looks good. Splitting it into two (MAPREDUCE-5181 for the mr part). The same needs to be done for the CLIENT secret. Opening a jira for this. > Make ApplicationToken part of Container's token list to help RM-restart > --- > > Key: YARN-579 > URL: https://issues.apache.org/jira/browse/YARN-579 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Attachments: YARN-579-20130422.1.txt > > > Container is already persisted for helping RM restart. Instead of explicitly > setting ApplicationToken in AM's env, if we change it to be in Container, we > can avoid env and can also help restart. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-579) Make ApplicationToken part of Container's token list to help RM-restart
[
https://issues.apache.org/jira/browse/YARN-579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13638703#comment-13638703
]
Hadoop QA commented on YARN-579:
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12579953/YARN-579-20130422.1.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/802//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/802//console
This message is automatically generated.
> Make ApplicationToken part of Container's token list to help RM-restart
> ---
>
> Key: YARN-579
> URL: https://issues.apache.org/jira/browse/YARN-579
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Vinod Kumar Vavilapalli
>Assignee: Vinod Kumar Vavilapalli
> Attachments: YARN-579-20130422.1.txt
>
>
> Container is already persisted for helping RM restart. Instead of explicitly
> setting ApplicationToken in AM's env, if we change it to be in Container, we
> can avoid env and can also help restart.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-579) Make ApplicationToken part of Container's token list to help RM-restart
[ https://issues.apache.org/jira/browse/YARN-579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13632992#comment-13632992 ] Bikas Saha commented on YARN-579: - In order to help restart, creation of appToken needs to move out from AMLauncher. AppToken needs to be set before the app attempt is saved. App attempt is saved after masterContainer is received from scheduler. > Make ApplicationToken part of Container's token list to help RM-restart > --- > > Key: YARN-579 > URL: https://issues.apache.org/jira/browse/YARN-579 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > > Container is already persisted for helping RM restart. Instead of explicitly > setting ApplicationToken in AM's env, if we change it to be in Container, we > can avoid env and can also help restart. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
