[jira] [Commented] (YARN-2003) Support to process Job priority from Submission Context in AppAttemptAddedSchedulerEvent [RM side]
[ https://issues.apache.org/jira/browse/YARN-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14599776#comment-14599776 ] Hadoop QA commented on YARN-2003: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | pre-patch | 21m 31s | Pre-patch trunk compilation is healthy. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | tests included | 0m 0s | The patch appears to include 8 new or modified test files. | | {color:green}+1{color} | javac | 10m 35s | There were no new javac warning messages. | | {color:red}-1{color} | javadoc | 11m 48s | The applied patch generated 1 additional warning messages. | | {color:green}+1{color} | release audit | 0m 26s | The applied patch does not increase the total number of release audit warnings. | | {color:red}-1{color} | checkstyle | 1m 22s | The applied patch generated 12 new checkstyle issues (total was 375, now 386). | | {color:red}-1{color} | whitespace | 0m 7s | The patch has 19 line(s) that end in whitespace. Use git apply --whitespace=fix. | | {color:green}+1{color} | install | 1m 45s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 35s | The patch built with eclipse:eclipse. | | {color:green}+1{color} | findbugs | 2m 45s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. | | {color:green}+1{color} | tools/hadoop tests | 0m 58s | Tests passed in hadoop-sls. | | {color:red}-1{color} | yarn tests | 52m 35s | Tests failed in hadoop-yarn-server-resourcemanager. | | | | 104m 33s | | \\ \\ || Reason || Tests || | Failed unit tests | hadoop.yarn.server.resourcemanager.scheduler.fair.TestSchedulingPolicy | | | hadoop.yarn.server.resourcemanager.security.TestRMDelegationTokens | | | hadoop.yarn.server.resourcemanager.TestAppManager | | Timed out tests | org.apache.hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12741636/0011-YARN-2003.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | trunk / 72d08a0 | | javadoc | https://builds.apache.org/job/PreCommit-YARN-Build/8332/artifact/patchprocess/diffJavadocWarnings.txt | | checkstyle | https://builds.apache.org/job/PreCommit-YARN-Build/8332/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt | | whitespace | https://builds.apache.org/job/PreCommit-YARN-Build/8332/artifact/patchprocess/whitespace.txt | | hadoop-sls test log | https://builds.apache.org/job/PreCommit-YARN-Build/8332/artifact/patchprocess/testrun_hadoop-sls.txt | | hadoop-yarn-server-resourcemanager test log | https://builds.apache.org/job/PreCommit-YARN-Build/8332/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt | | Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/8332/testReport/ | | Java | 1.7.0_55 | | uname | Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Console output | https://builds.apache.org/job/PreCommit-YARN-Build/8332/console | This message was automatically generated. > Support to process Job priority from Submission Context in > AppAttemptAddedSchedulerEvent [RM side] > -- > > Key: YARN-2003 > URL: https://issues.apache.org/jira/browse/YARN-2003 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Labels: BB2015-05-TBR > Attachments: 0001-YARN-2003.patch, 00010-YARN-2003.patch, > 0002-YARN-2003.patch, 0003-YARN-2003.patch, 0004-YARN-2003.patch, > 0005-YARN-2003.patch, 0006-YARN-2003.patch, 0007-YARN-2003.patch, > 0008-YARN-2003.patch, 0009-YARN-2003.patch, 0011-YARN-2003.patch > > > AppAttemptAddedSchedulerEvent should be able to receive the Job Priority from > Submission Context and store. > Later this can be used by Scheduler. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2003) Support to process Job priority from Submission Context in AppAttemptAddedSchedulerEvent [RM side]
[ https://issues.apache.org/jira/browse/YARN-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14600044#comment-14600044 ] Hadoop QA commented on YARN-2003: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:red}-1{color} | pre-patch | 15m 46s | Findbugs (version ) appears to be broken on trunk. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | tests included | 0m 0s | The patch appears to include 8 new or modified test files. | | {color:green}+1{color} | javac | 7m 36s | There were no new javac warning messages. | | {color:green}+1{color} | javadoc | 9m 40s | There were no new javadoc warning messages. | | {color:green}+1{color} | release audit | 0m 23s | The applied patch does not increase the total number of release audit warnings. | | {color:green}+1{color} | checkstyle | 0m 43s | There were no new checkstyle issues. | | {color:red}-1{color} | whitespace | 0m 6s | The patch has 18 line(s) that end in whitespace. Use git apply --whitespace=fix. | | {color:green}+1{color} | install | 1m 35s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 34s | The patch built with eclipse:eclipse. | | {color:green}+1{color} | findbugs | 2m 18s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. | | {color:green}+1{color} | tools/hadoop tests | 0m 52s | Tests passed in hadoop-sls. | | {color:green}+1{color} | yarn tests | 51m 0s | Tests passed in hadoop-yarn-server-resourcemanager. | | | | 90m 37s | | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12741671/0012-YARN-2003.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | trunk / 2df00d5 | | whitespace | https://builds.apache.org/job/PreCommit-YARN-Build/8333/artifact/patchprocess/whitespace.txt | | hadoop-sls test log | https://builds.apache.org/job/PreCommit-YARN-Build/8333/artifact/patchprocess/testrun_hadoop-sls.txt | | hadoop-yarn-server-resourcemanager test log | https://builds.apache.org/job/PreCommit-YARN-Build/8333/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt | | Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/8333/testReport/ | | Java | 1.7.0_55 | | uname | Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Console output | https://builds.apache.org/job/PreCommit-YARN-Build/8333/console | This message was automatically generated. > Support to process Job priority from Submission Context in > AppAttemptAddedSchedulerEvent [RM side] > -- > > Key: YARN-2003 > URL: https://issues.apache.org/jira/browse/YARN-2003 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Labels: BB2015-05-TBR > Attachments: 0001-YARN-2003.patch, 00010-YARN-2003.patch, > 0002-YARN-2003.patch, 0003-YARN-2003.patch, 0004-YARN-2003.patch, > 0005-YARN-2003.patch, 0006-YARN-2003.patch, 0007-YARN-2003.patch, > 0008-YARN-2003.patch, 0009-YARN-2003.patch, 0011-YARN-2003.patch, > 0012-YARN-2003.patch > > > AppAttemptAddedSchedulerEvent should be able to receive the Job Priority from > Submission Context and store. > Later this can be used by Scheduler. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2003) Support to process Job priority from Submission Context in AppAttemptAddedSchedulerEvent [RM side]
[ https://issues.apache.org/jira/browse/YARN-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14603425#comment-14603425 ] Wangda Tan commented on YARN-2003: -- Hi [~sunilg], Thanks for updating, some comments: 1) API for YARNScheduler: - updateApplicationPriority, we don't need pass user/queueName, since scheduler should know it. authenticate is different, since scheduler may not have application information at that time. - It may be better to throw YarnException instead of IOException. 2) RMAppManager: - Is this check necessary? {{rmContext.getScheduler() != null}}, if this is for test cases, I think it's better to fix tests. > Support to process Job priority from Submission Context in > AppAttemptAddedSchedulerEvent [RM side] > -- > > Key: YARN-2003 > URL: https://issues.apache.org/jira/browse/YARN-2003 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Labels: BB2015-05-TBR > Attachments: 0001-YARN-2003.patch, 00010-YARN-2003.patch, > 0002-YARN-2003.patch, 0003-YARN-2003.patch, 0004-YARN-2003.patch, > 0005-YARN-2003.patch, 0006-YARN-2003.patch, 0007-YARN-2003.patch, > 0008-YARN-2003.patch, 0009-YARN-2003.patch, 0011-YARN-2003.patch, > 0012-YARN-2003.patch > > > AppAttemptAddedSchedulerEvent should be able to receive the Job Priority from > Submission Context and store. > Later this can be used by Scheduler. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2003) Support to process Job priority from Submission Context in AppAttemptAddedSchedulerEvent [RM side]
[ https://issues.apache.org/jira/browse/YARN-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14604689#comment-14604689 ] Hadoop QA commented on YARN-2003: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | pre-patch | 18m 9s | Pre-patch trunk compilation is healthy. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | tests included | 0m 0s | The patch appears to include 9 new or modified test files. | | {color:red}-1{color} | javac | 8m 1s | The patch appears to cause the build to fail. | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12742391/0013-YARN-2003.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | trunk / b543d1a | | Console output | https://builds.apache.org/job/PreCommit-YARN-Build/8368/console | This message was automatically generated. > Support to process Job priority from Submission Context in > AppAttemptAddedSchedulerEvent [RM side] > -- > > Key: YARN-2003 > URL: https://issues.apache.org/jira/browse/YARN-2003 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Labels: BB2015-05-TBR > Attachments: 0001-YARN-2003.patch, 00010-YARN-2003.patch, > 0002-YARN-2003.patch, 0003-YARN-2003.patch, 0004-YARN-2003.patch, > 0005-YARN-2003.patch, 0006-YARN-2003.patch, 0007-YARN-2003.patch, > 0008-YARN-2003.patch, 0009-YARN-2003.patch, 0011-YARN-2003.patch, > 0012-YARN-2003.patch, 0013-YARN-2003.patch > > > AppAttemptAddedSchedulerEvent should be able to receive the Job Priority from > Submission Context and store. > Later this can be used by Scheduler. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2003) Support to process Job priority from Submission Context in AppAttemptAddedSchedulerEvent [RM side]
[ https://issues.apache.org/jira/browse/YARN-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14604757#comment-14604757 ] Hadoop QA commented on YARN-2003: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | pre-patch | 16m 34s | Pre-patch trunk compilation is healthy. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | tests included | 0m 0s | The patch appears to include 9 new or modified test files. | | {color:green}+1{color} | javac | 7m 35s | There were no new javac warning messages. | | {color:green}+1{color} | javadoc | 9m 39s | There were no new javadoc warning messages. | | {color:green}+1{color} | release audit | 0m 22s | The applied patch does not increase the total number of release audit warnings. | | {color:red}-1{color} | checkstyle | 1m 3s | The applied patch generated 10 new checkstyle issues (total was 375, now 382). | | {color:red}-1{color} | whitespace | 0m 6s | The patch has 18 line(s) that end in whitespace. Use git apply --whitespace=fix. | | {color:green}+1{color} | install | 1m 35s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with eclipse:eclipse. | | {color:green}+1{color} | findbugs | 2m 14s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. | | {color:green}+1{color} | tools/hadoop tests | 0m 53s | Tests passed in hadoop-sls. | | {color:green}+1{color} | yarn tests | 50m 55s | Tests passed in hadoop-yarn-server-resourcemanager. | | | | 91m 33s | | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12742395/0014-YARN-2003.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | trunk / b543d1a | | checkstyle | https://builds.apache.org/job/PreCommit-YARN-Build/8369/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt | | whitespace | https://builds.apache.org/job/PreCommit-YARN-Build/8369/artifact/patchprocess/whitespace.txt | | hadoop-sls test log | https://builds.apache.org/job/PreCommit-YARN-Build/8369/artifact/patchprocess/testrun_hadoop-sls.txt | | hadoop-yarn-server-resourcemanager test log | https://builds.apache.org/job/PreCommit-YARN-Build/8369/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt | | Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/8369/testReport/ | | Java | 1.7.0_55 | | uname | Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Console output | https://builds.apache.org/job/PreCommit-YARN-Build/8369/console | This message was automatically generated. > Support to process Job priority from Submission Context in > AppAttemptAddedSchedulerEvent [RM side] > -- > > Key: YARN-2003 > URL: https://issues.apache.org/jira/browse/YARN-2003 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Labels: BB2015-05-TBR > Attachments: 0001-YARN-2003.patch, 00010-YARN-2003.patch, > 0002-YARN-2003.patch, 0003-YARN-2003.patch, 0004-YARN-2003.patch, > 0005-YARN-2003.patch, 0006-YARN-2003.patch, 0007-YARN-2003.patch, > 0008-YARN-2003.patch, 0009-YARN-2003.patch, 0011-YARN-2003.patch, > 0012-YARN-2003.patch, 0013-YARN-2003.patch, 0014-YARN-2003.patch > > > AppAttemptAddedSchedulerEvent should be able to receive the Job Priority from > Submission Context and store. > Later this can be used by Scheduler. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2003) Support to process Job priority from Submission Context in AppAttemptAddedSchedulerEvent [RM side]
[ https://issues.apache.org/jira/browse/YARN-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14608409#comment-14608409 ] Hadoop QA commented on YARN-2003: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | pre-patch | 17m 59s | Pre-patch trunk compilation is healthy. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | tests included | 0m 0s | The patch appears to include 9 new or modified test files. | | {color:red}-1{color} | javac | 3m 24s | The patch appears to cause the build to fail. | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12742842/0015-YARN-2003.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | trunk / 076948d | | Console output | https://builds.apache.org/job/PreCommit-YARN-Build/8387/console | This message was automatically generated. > Support to process Job priority from Submission Context in > AppAttemptAddedSchedulerEvent [RM side] > -- > > Key: YARN-2003 > URL: https://issues.apache.org/jira/browse/YARN-2003 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Labels: BB2015-05-TBR > Attachments: 0001-YARN-2003.patch, 00010-YARN-2003.patch, > 0002-YARN-2003.patch, 0003-YARN-2003.patch, 0004-YARN-2003.patch, > 0005-YARN-2003.patch, 0006-YARN-2003.patch, 0007-YARN-2003.patch, > 0008-YARN-2003.patch, 0009-YARN-2003.patch, 0011-YARN-2003.patch, > 0012-YARN-2003.patch, 0013-YARN-2003.patch, 0014-YARN-2003.patch, > 0015-YARN-2003.patch > > > AppAttemptAddedSchedulerEvent should be able to receive the Job Priority from > Submission Context and store. > Later this can be used by Scheduler. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2003) Support to process Job priority from Submission Context in AppAttemptAddedSchedulerEvent [RM side]
[ https://issues.apache.org/jira/browse/YARN-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14608741#comment-14608741 ] Hadoop QA commented on YARN-2003: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | pre-patch | 16m 43s | Pre-patch trunk compilation is healthy. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | tests included | 0m 0s | The patch appears to include 9 new or modified test files. | | {color:green}+1{color} | javac | 7m 36s | There were no new javac warning messages. | | {color:green}+1{color} | javadoc | 9m 39s | There were no new javadoc warning messages. | | {color:green}+1{color} | release audit | 0m 21s | The applied patch does not increase the total number of release audit warnings. | | {color:red}-1{color} | checkstyle | 1m 5s | The applied patch generated 13 new checkstyle issues (total was 377, now 387). | | {color:red}-1{color} | whitespace | 0m 8s | The patch has 19 line(s) that end in whitespace. Use git apply --whitespace=fix. | | {color:green}+1{color} | install | 1m 34s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 31s | The patch built with eclipse:eclipse. | | {color:green}+1{color} | findbugs | 2m 14s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. | | {color:green}+1{color} | tools/hadoop tests | 0m 51s | Tests passed in hadoop-sls. | | {color:green}+1{color} | yarn tests | 50m 45s | Tests passed in hadoop-yarn-server-resourcemanager. | | | | 91m 30s | | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12742863/0015-YARN-2003.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | trunk / 076948d | | checkstyle | https://builds.apache.org/job/PreCommit-YARN-Build/8390/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt | | whitespace | https://builds.apache.org/job/PreCommit-YARN-Build/8390/artifact/patchprocess/whitespace.txt | | hadoop-sls test log | https://builds.apache.org/job/PreCommit-YARN-Build/8390/artifact/patchprocess/testrun_hadoop-sls.txt | | hadoop-yarn-server-resourcemanager test log | https://builds.apache.org/job/PreCommit-YARN-Build/8390/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt | | Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/8390/testReport/ | | Java | 1.7.0_55 | | uname | Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Console output | https://builds.apache.org/job/PreCommit-YARN-Build/8390/console | This message was automatically generated. > Support to process Job priority from Submission Context in > AppAttemptAddedSchedulerEvent [RM side] > -- > > Key: YARN-2003 > URL: https://issues.apache.org/jira/browse/YARN-2003 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Labels: BB2015-05-TBR > Attachments: 0001-YARN-2003.patch, 00010-YARN-2003.patch, > 0002-YARN-2003.patch, 0003-YARN-2003.patch, 0004-YARN-2003.patch, > 0005-YARN-2003.patch, 0006-YARN-2003.patch, 0007-YARN-2003.patch, > 0008-YARN-2003.patch, 0009-YARN-2003.patch, 0011-YARN-2003.patch, > 0012-YARN-2003.patch, 0013-YARN-2003.patch, 0014-YARN-2003.patch, > 0015-YARN-2003.patch > > > AppAttemptAddedSchedulerEvent should be able to receive the Job Priority from > Submission Context and store. > Later this can be used by Scheduler. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2003) Support to process Job priority from Submission Context in AppAttemptAddedSchedulerEvent [RM side]
[ https://issues.apache.org/jira/browse/YARN-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14389777#comment-14389777 ] Wangda Tan commented on YARN-2003: -- [~sunilg], patch is straight forward to me, could you rebase YARN-2004 patch based on this, and according to our discussion, do you think it's reasonable to add an update-application-priority API to scheduler? > Support to process Job priority from Submission Context in > AppAttemptAddedSchedulerEvent [RM side] > -- > > Key: YARN-2003 > URL: https://issues.apache.org/jira/browse/YARN-2003 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Attachments: 0001-YARN-2003.patch, 0002-YARN-2003.patch, > 0003-YARN-2003.patch, 0004-YARN-2003.patch, 0005-YARN-2003.patch > > > AppAttemptAddedSchedulerEvent should be able to receive the Job Priority from > Submission Context and store. > Later this can be used by Scheduler. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2003) Support to process Job priority from Submission Context in AppAttemptAddedSchedulerEvent [RM side]
[ https://issues.apache.org/jira/browse/YARN-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14390995#comment-14390995 ] Sunil G commented on YARN-2003: --- Thank You [~leftnoteasy] I will rebase YARN-2004 and will share a patch short while . bq.do you think it's reasonable to add an update-application-priority API to scheduler? Yes. We can add that. I will update this patch with same. > Support to process Job priority from Submission Context in > AppAttemptAddedSchedulerEvent [RM side] > -- > > Key: YARN-2003 > URL: https://issues.apache.org/jira/browse/YARN-2003 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Attachments: 0001-YARN-2003.patch, 0002-YARN-2003.patch, > 0003-YARN-2003.patch, 0004-YARN-2003.patch, 0005-YARN-2003.patch > > > AppAttemptAddedSchedulerEvent should be able to receive the Job Priority from > Submission Context and store. > Later this can be used by Scheduler. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2003) Support to process Job priority from Submission Context in AppAttemptAddedSchedulerEvent [RM side]
[ https://issues.apache.org/jira/browse/YARN-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14394891#comment-14394891 ] Hadoop QA commented on YARN-2003: - {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12708208/0005-YARN-2003.patch against trunk revision db80e42. {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 7 new or modified test files. {color:red}-1 javac{color:red}. The patch appears to cause the build to fail. Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7213//console This message is automatically generated. > Support to process Job priority from Submission Context in > AppAttemptAddedSchedulerEvent [RM side] > -- > > Key: YARN-2003 > URL: https://issues.apache.org/jira/browse/YARN-2003 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Attachments: 0001-YARN-2003.patch, 0002-YARN-2003.patch, > 0003-YARN-2003.patch, 0004-YARN-2003.patch, 0005-YARN-2003.patch > > > AppAttemptAddedSchedulerEvent should be able to receive the Job Priority from > Submission Context and store. > Later this can be used by Scheduler. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2003) Support to process Job priority from Submission Context in AppAttemptAddedSchedulerEvent [RM side]
[ https://issues.apache.org/jira/browse/YARN-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14483416#comment-14483416 ] Hadoop QA commented on YARN-2003: - {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12723643/0006-YARN-2003.patch against trunk revision 75c5454. {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 7 new or modified test files. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javadoc{color}. There were no new javadoc warning messages. {color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse. {color:red}-1 findbugs{color}. The patch appears to introduce 13 new Findbugs (version 2.0.3) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 core tests{color}. The patch passed unit tests in hadoop-tools/hadoop-sls hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/7236//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-YARN-Build/7236//artifact/patchprocess/newPatchFindbugsWarningshadoop-sls.html Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7236//console This message is automatically generated. > Support to process Job priority from Submission Context in > AppAttemptAddedSchedulerEvent [RM side] > -- > > Key: YARN-2003 > URL: https://issues.apache.org/jira/browse/YARN-2003 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Attachments: 0001-YARN-2003.patch, 0002-YARN-2003.patch, > 0003-YARN-2003.patch, 0004-YARN-2003.patch, 0005-YARN-2003.patch, > 0006-YARN-2003.patch > > > AppAttemptAddedSchedulerEvent should be able to receive the Job Priority from > Submission Context and store. > Later this can be used by Scheduler. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2003) Support to process Job priority from Submission Context in AppAttemptAddedSchedulerEvent [RM side]
[ https://issues.apache.org/jira/browse/YARN-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14485392#comment-14485392 ] Sunil G commented on YARN-2003: --- Findbugs warnings are not related. > Support to process Job priority from Submission Context in > AppAttemptAddedSchedulerEvent [RM side] > -- > > Key: YARN-2003 > URL: https://issues.apache.org/jira/browse/YARN-2003 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Attachments: 0001-YARN-2003.patch, 0002-YARN-2003.patch, > 0003-YARN-2003.patch, 0004-YARN-2003.patch, 0005-YARN-2003.patch, > 0006-YARN-2003.patch > > > AppAttemptAddedSchedulerEvent should be able to receive the Job Priority from > Submission Context and store. > Later this can be used by Scheduler. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2003) Support to process Job priority from Submission Context in AppAttemptAddedSchedulerEvent [RM side]
[ https://issues.apache.org/jira/browse/YARN-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14499009#comment-14499009 ] Wangda Tan commented on YARN-2003: -- Another comment is about authenticateApplicationPriority, I'm wondering if we really need it. Some other fields like ACL, queue state, etc. are checked in LeafQueue.submitApplication. So I think queue app priority setting may be better handled within queue. Thoughts? > Support to process Job priority from Submission Context in > AppAttemptAddedSchedulerEvent [RM side] > -- > > Key: YARN-2003 > URL: https://issues.apache.org/jira/browse/YARN-2003 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Attachments: 0001-YARN-2003.patch, 0002-YARN-2003.patch, > 0003-YARN-2003.patch, 0004-YARN-2003.patch, 0005-YARN-2003.patch, > 0006-YARN-2003.patch > > > AppAttemptAddedSchedulerEvent should be able to receive the Job Priority from > Submission Context and store. > Later this can be used by Scheduler. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2003) Support to process Job priority from Submission Context in AppAttemptAddedSchedulerEvent [RM side]
[ https://issues.apache.org/jira/browse/YARN-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14498992#comment-14498992 ] Wangda Tan commented on YARN-2003: -- Some comments: 1) Don't need to change Leaf/ParentQueue, change AbstractCSQueue should be enough. As same as Fair queue 2) How about adding an update-application-priority API to scheduler? 3) Patch need rebase > Support to process Job priority from Submission Context in > AppAttemptAddedSchedulerEvent [RM side] > -- > > Key: YARN-2003 > URL: https://issues.apache.org/jira/browse/YARN-2003 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Attachments: 0001-YARN-2003.patch, 0002-YARN-2003.patch, > 0003-YARN-2003.patch, 0004-YARN-2003.patch, 0005-YARN-2003.patch, > 0006-YARN-2003.patch > > > AppAttemptAddedSchedulerEvent should be able to receive the Job Priority from > Submission Context and store. > Later this can be used by Scheduler. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2003) Support to process Job priority from Submission Context in AppAttemptAddedSchedulerEvent [RM side]
[ https://issues.apache.org/jira/browse/YARN-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14499401#comment-14499401 ] Sunil G commented on YARN-2003: --- HI [~leftnoteasy] bq. authenticateApplicationPriority, I'm wondering if we really need it. I understand your point. But we may fire a new APP_ADDED event from RmAppManager to respective scheduler with a unapproved priority, and then reject from there. If we can do much earlier with help of a single api, it may avoid some extra event handling in the case of wrong(invalid priority) app submission. > Support to process Job priority from Submission Context in > AppAttemptAddedSchedulerEvent [RM side] > -- > > Key: YARN-2003 > URL: https://issues.apache.org/jira/browse/YARN-2003 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Attachments: 0001-YARN-2003.patch, 0002-YARN-2003.patch, > 0003-YARN-2003.patch, 0004-YARN-2003.patch, 0005-YARN-2003.patch, > 0006-YARN-2003.patch > > > AppAttemptAddedSchedulerEvent should be able to receive the Job Priority from > Submission Context and store. > Later this can be used by Scheduler. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2003) Support to process Job priority from Submission Context in AppAttemptAddedSchedulerEvent [RM side]
[ https://issues.apache.org/jira/browse/YARN-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14499402#comment-14499402 ] Sunil G commented on YARN-2003: --- Thank you [~leftnoteasy] for sharing comments. I will rebase patch and will address the comments mentioned in above comment > Support to process Job priority from Submission Context in > AppAttemptAddedSchedulerEvent [RM side] > -- > > Key: YARN-2003 > URL: https://issues.apache.org/jira/browse/YARN-2003 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Attachments: 0001-YARN-2003.patch, 0002-YARN-2003.patch, > 0003-YARN-2003.patch, 0004-YARN-2003.patch, 0005-YARN-2003.patch, > 0006-YARN-2003.patch > > > AppAttemptAddedSchedulerEvent should be able to receive the Job Priority from > Submission Context and store. > Later this can be used by Scheduler. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2003) Support to process Job priority from Submission Context in AppAttemptAddedSchedulerEvent [RM side]
[ https://issues.apache.org/jira/browse/YARN-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14500417#comment-14500417 ] Hadoop QA commented on YARN-2003: - {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12726222/0007-YARN-2003.patch against trunk revision c6b5203. {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 7 new or modified test files. {color:red}-1 javac{color:red}. The patch appears to cause the build to fail. Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7382//console This message is automatically generated. > Support to process Job priority from Submission Context in > AppAttemptAddedSchedulerEvent [RM side] > -- > > Key: YARN-2003 > URL: https://issues.apache.org/jira/browse/YARN-2003 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Attachments: 0001-YARN-2003.patch, 0002-YARN-2003.patch, > 0003-YARN-2003.patch, 0004-YARN-2003.patch, 0005-YARN-2003.patch, > 0006-YARN-2003.patch, 0007-YARN-2003.patch > > > AppAttemptAddedSchedulerEvent should be able to receive the Job Priority from > Submission Context and store. > Later this can be used by Scheduler. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2003) Support to process Job priority from Submission Context in AppAttemptAddedSchedulerEvent [RM side]
[ https://issues.apache.org/jira/browse/YARN-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14500513#comment-14500513 ] Hadoop QA commented on YARN-2003: - {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12726239/0008-YARN-2003.patch against trunk revision c6b5203. {color:red}-1 patch{color}. Trunk compilation may be broken. Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7384//console This message is automatically generated. > Support to process Job priority from Submission Context in > AppAttemptAddedSchedulerEvent [RM side] > -- > > Key: YARN-2003 > URL: https://issues.apache.org/jira/browse/YARN-2003 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Attachments: 0001-YARN-2003.patch, 0002-YARN-2003.patch, > 0003-YARN-2003.patch, 0004-YARN-2003.patch, 0005-YARN-2003.patch, > 0006-YARN-2003.patch, 0007-YARN-2003.patch, 0008-YARN-2003.patch > > > AppAttemptAddedSchedulerEvent should be able to receive the Job Priority from > Submission Context and store. > Later this can be used by Scheduler. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2003) Support to process Job priority from Submission Context in AppAttemptAddedSchedulerEvent [RM side]
[ https://issues.apache.org/jira/browse/YARN-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14501344#comment-14501344 ] Hadoop QA commented on YARN-2003: - {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12726353/0009-YARN-2003.patch against trunk revision f47a576. {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 8 new or modified test files. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1 warning messages. See https://builds.apache.org/job/PreCommit-YARN-Build/7390//artifact/patchprocess/diffJavadocWarnings.txt for details. {color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse. {color:red}-1 findbugs{color}. The patch appears to introduce 13 new Findbugs (version 2.0.3) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 core tests{color}. The patch passed unit tests in hadoop-tools/hadoop-sls hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/7390//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-YARN-Build/7390//artifact/patchprocess/newPatchFindbugsWarningshadoop-sls.html Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7390//console This message is automatically generated. > Support to process Job priority from Submission Context in > AppAttemptAddedSchedulerEvent [RM side] > -- > > Key: YARN-2003 > URL: https://issues.apache.org/jira/browse/YARN-2003 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Attachments: 0001-YARN-2003.patch, 0002-YARN-2003.patch, > 0003-YARN-2003.patch, 0004-YARN-2003.patch, 0005-YARN-2003.patch, > 0006-YARN-2003.patch, 0007-YARN-2003.patch, 0008-YARN-2003.patch, > 0009-YARN-2003.patch > > > AppAttemptAddedSchedulerEvent should be able to receive the Job Priority from > Submission Context and store. > Later this can be used by Scheduler. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2003) Support to process Job priority from Submission Context in AppAttemptAddedSchedulerEvent [RM side]
[ https://issues.apache.org/jira/browse/YARN-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14501517#comment-14501517 ] Sunil G commented on YARN-2003: --- Findbugs warnings are not related. > Support to process Job priority from Submission Context in > AppAttemptAddedSchedulerEvent [RM side] > -- > > Key: YARN-2003 > URL: https://issues.apache.org/jira/browse/YARN-2003 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Attachments: 0001-YARN-2003.patch, 0002-YARN-2003.patch, > 0003-YARN-2003.patch, 0004-YARN-2003.patch, 0005-YARN-2003.patch, > 0006-YARN-2003.patch, 0007-YARN-2003.patch, 0008-YARN-2003.patch, > 0009-YARN-2003.patch > > > AppAttemptAddedSchedulerEvent should be able to receive the Job Priority from > Submission Context and store. > Later this can be used by Scheduler. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2003) Support to process Job priority from Submission Context in AppAttemptAddedSchedulerEvent [RM side]
[ https://issues.apache.org/jira/browse/YARN-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14503507#comment-14503507 ] Hadoop QA commented on YARN-2003: - {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12726620/00010-YARN-2003.patch against trunk revision f967fd2. {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 7 new or modified test files. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javadoc{color}. There were no new javadoc warning messages. {color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 2.0.3) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 core tests{color}. The patch failed these unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerQueueACLs org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesFairScheduler org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification org.apache.hadoop.yarn.server.resourcemanager.security.TestAMRMTokens org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySched org.apache.hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStoreZKClientConnections org.apache.hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStore org.apache.hadoop.yarn.server.resourcemanager.recovery.TestLeveldbRMStateStore org.apache.hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStorePerf org.apache.hadoop.yarn.server.resourcemanager.security.TestClientToAMTokens org.apache.hadoop.yarn.server.resourcemanager.recovery.TestFSRMStateStore org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization The following test timeouts occurred in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: org.apache.hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRTestTests org.apache.hadoop.yarn.server.resourcemanager.TestRMRestTestTests org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorizatTestTests Test results: https://builds.apache.org/job/PreCommit-YARN-Build/7400//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7400//console This message is automatically generated. > Support to process Job priority from Submission Context in > AppAttemptAddedSchedulerEvent [RM side] > -- > > Key: YARN-2003 > URL: https://issues.apache.org/jira/browse/YARN-2003 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Attachments: 0001-YARN-2003.patch, 00010-YARN-2003.patch, > 0002-YARN-2003.patch, 0003-YARN-2003.patch, 0004-YARN-2003.patch, > 0005-YARN-2003.patch, 0006-YARN-2003.patch, 0007-YARN-2003.patch, > 0008-YARN-2003.patch, 0009-YARN-2003.patch > > > AppAttemptAddedSchedulerEvent should be able to receive the Job Priority from > Submission Context and store. > Later this can be used by Scheduler. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2003) Support to process Job priority from Submission Context in AppAttemptAddedSchedulerEvent [RM side]
[ https://issues.apache.org/jira/browse/YARN-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14243145#comment-14243145 ] Ashwin Shankar commented on YARN-2003: -- Hi [~sunilg], is there anything that needs to be done on the RM side for app priority to work in fair scheduler ? There is already a patch for app priority from the fair sched side and was wondering if anything in this jira is blocking it. > Support to process Job priority from Submission Context in > AppAttemptAddedSchedulerEvent [RM side] > -- > > Key: YARN-2003 > URL: https://issues.apache.org/jira/browse/YARN-2003 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Sunil G >Assignee: Sunil G > > AppAttemptAddedSchedulerEvent should be able to receive the Job Priority from > Submission Context and store. > Later this can be used by Scheduler. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2003) Support to process Job priority from Submission Context in AppAttemptAddedSchedulerEvent [RM side]
[ https://issues.apache.org/jira/browse/YARN-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14243618#comment-14243618 ] Sunil G commented on YARN-2003: --- Hi [~ashwinshankar77] for fair and capacity scheduler, we were planning to keep the PrioirtyLabel Manager as common, But internally each scheduler has to handle the app priority separately. And outline patch only is available now, but once the initial set of patches of Priority Label Manager is checked in, this patch can be updated to work for fair scheduler. > Support to process Job priority from Submission Context in > AppAttemptAddedSchedulerEvent [RM side] > -- > > Key: YARN-2003 > URL: https://issues.apache.org/jira/browse/YARN-2003 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Sunil G >Assignee: Sunil G > > AppAttemptAddedSchedulerEvent should be able to receive the Job Priority from > Submission Context and store. > Later this can be used by Scheduler. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2003) Support to process Job priority from Submission Context in AppAttemptAddedSchedulerEvent [RM side]
[ https://issues.apache.org/jira/browse/YARN-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14244854#comment-14244854 ] Ashwin Shankar commented on YARN-2003: -- [~sunilg], Thanks for letting me know. Just wanted to emphasis that we are really interested in getting app priorities for fair scheduler committed soon, since we have few usecases in our production cluster that need this. Please let me know if you need help with the common code between the schedulers. > Support to process Job priority from Submission Context in > AppAttemptAddedSchedulerEvent [RM side] > -- > > Key: YARN-2003 > URL: https://issues.apache.org/jira/browse/YARN-2003 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Sunil G >Assignee: Sunil G > > AppAttemptAddedSchedulerEvent should be able to receive the Job Priority from > Submission Context and store. > Later this can be used by Scheduler. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2003) Support to process Job priority from Submission Context in AppAttemptAddedSchedulerEvent [RM side]
[ https://issues.apache.org/jira/browse/YARN-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13985340#comment-13985340 ] Sunil G commented on YARN-2003: --- ApplicationSubmissionContext already has an implementation for Priority. I am reusing the same code, hence no protocol changes are required. > Support to process Job priority from Submission Context in > AppAttemptAddedSchedulerEvent [RM side] > -- > > Key: YARN-2003 > URL: https://issues.apache.org/jira/browse/YARN-2003 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Sunil G > > AppAttemptAddedSchedulerEvent should be able to receive the Job Priority from > Submission Context and store. > Later this can be used by Scheduler. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-2003) Support to process Job priority from Submission Context in AppAttemptAddedSchedulerEvent [RM side]
[ https://issues.apache.org/jira/browse/YARN-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14366273#comment-14366273 ] Wangda Tan commented on YARN-2003: -- Hi [~sunilg], Thanks for working on this. I took a quick look at YARN-2003 / YARN-2004. Some overall comments. First I want to describe my thinkings of how RM side workflow look like: - When application submit to RMAppManager, it will simply pass priority set by user to RMApp (see (1)) - RMApp will finally create APP_ATTEMPT_ADDED, queue itself will normalize priority (reject it / convert it from label to number, etc. And set new priority to RMApp), set priority to SchedulerApplicationAttempt - Scheduler uses priority in SchedulerApplicationAttempt and Queue to make scheduling decisions - If user ask for changing application priority, or admin changes priority configuration, event may need send to scheduler to update inner applications/queues. - When user requires priority of application via CLI/Web-UI, ApplicationPriorityManager will convert number to label (if possible) and show to user. (1). We don't have to add too much logic here, and if we can simply handle it inside scheduler, when configuration changed (like label->integer priority mapping changed), it can be handled by scheduler itself. Back to your patches, several major differences are: - RMAppManager takes responsibility to check ACL for priority, which I think not proper (ACL is always managed by scheduler queues) and not correct (when queue configuration changed, you cannot do rechecking application priority if new app attempt created.). - As same as above, queues should check ACLs for priority. - You may not need add priority to SchedulerApplication, it seems not necessary to me. > Support to process Job priority from Submission Context in > AppAttemptAddedSchedulerEvent [RM side] > -- > > Key: YARN-2003 > URL: https://issues.apache.org/jira/browse/YARN-2003 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Attachments: 0001-YARN-2003.patch, 0002-YARN-2003.patch, > 0003-YARN-2003.patch, 0004-YARN-2003.patch > > > AppAttemptAddedSchedulerEvent should be able to receive the Job Priority from > Submission Context and store. > Later this can be used by Scheduler. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2003) Support to process Job priority from Submission Context in AppAttemptAddedSchedulerEvent [RM side]
[ https://issues.apache.org/jira/browse/YARN-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14366282#comment-14366282 ] Wangda Tan commented on YARN-2003: -- In beyond, I suggest you can divide YARN-2003/YARN-2004 to be: YARN-2003: - Changes of RM side - Changes of scheduler Queue interface (May need some empty implementations in specific schedulers). - New scheduler event YARN-2003 should be able to compile/test without YARN-2004 YARN-2004 tracks changes only in CapacityScheduler side and need YARN-2004 to compile/test. Which this, I can simply apply two patches to do review. > Support to process Job priority from Submission Context in > AppAttemptAddedSchedulerEvent [RM side] > -- > > Key: YARN-2003 > URL: https://issues.apache.org/jira/browse/YARN-2003 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Attachments: 0001-YARN-2003.patch, 0002-YARN-2003.patch, > 0003-YARN-2003.patch, 0004-YARN-2003.patch > > > AppAttemptAddedSchedulerEvent should be able to receive the Job Priority from > Submission Context and store. > Later this can be used by Scheduler. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2003) Support to process Job priority from Submission Context in AppAttemptAddedSchedulerEvent [RM side]
[ https://issues.apache.org/jira/browse/YARN-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14367465#comment-14367465 ] Sunil G commented on YARN-2003: --- Thank you [~leftnoteasy] for sharing the comments Yes, YARN-2003 will focus on RM related changes excluding changes from Scheduler. I will rearrange code as per same and update. > Support to process Job priority from Submission Context in > AppAttemptAddedSchedulerEvent [RM side] > -- > > Key: YARN-2003 > URL: https://issues.apache.org/jira/browse/YARN-2003 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Sunil G >Assignee: Sunil G > Attachments: 0001-YARN-2003.patch, 0002-YARN-2003.patch, > 0003-YARN-2003.patch, 0004-YARN-2003.patch > > > AppAttemptAddedSchedulerEvent should be able to receive the Job Priority from > Submission Context and store. > Later this can be used by Scheduler. -- This message was sent by Atlassian JIRA (v6.3.4#6332)