[jira] [Commented] (YARN-2575) Consider creating separate ACLs for Reservation create/update/delete/list ops
[ https://issues.apache.org/jira/browse/YARN-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15143145#comment-15143145 ] Sean Po commented on YARN-2575: --- I double checked the hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA test failure by running the test locally with the patch applied on trunk and the test passed. YARN-4312 documents this test failure in the past, the fix is not currently applied in trunk. > Consider creating separate ACLs for Reservation create/update/delete/list ops > - > > Key: YARN-2575 > URL: https://issues.apache.org/jira/browse/YARN-2575 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, fairscheduler, resourcemanager >Reporter: Subru Krishnan >Assignee: Sean Po > Attachments: YARN-2575.v1.patch, YARN-2575.v10.patch, > YARN-2575.v11.patch, YARN-2575.v2.1.patch, YARN-2575.v2.patch, > YARN-2575.v3.patch, YARN-2575.v4.patch, YARN-2575.v5.patch, > YARN-2575.v6.patch, YARN-2575.v7.patch, YARN-2575.v8.patch, YARN-2575.v9.patch > > > YARN-1051 introduces the ReservationSystem and in the current implementation > anyone who can submit applications can also submit reservations. This JIRA is > to evaluate creating separate ACLs for Reservation create/update/delete ops. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2575) Consider creating separate ACLs for Reservation create/update/delete/list ops
[ https://issues.apache.org/jira/browse/YARN-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15143236#comment-15143236 ] Arun Suresh commented on YARN-2575: --- +1 Thanks for all the work on this [~seanpo03] and for the reviews [~subru]. Will commit this shortly > Consider creating separate ACLs for Reservation create/update/delete/list ops > - > > Key: YARN-2575 > URL: https://issues.apache.org/jira/browse/YARN-2575 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, fairscheduler, resourcemanager >Reporter: Subru Krishnan >Assignee: Sean Po > Attachments: YARN-2575.v1.patch, YARN-2575.v10.patch, > YARN-2575.v11.patch, YARN-2575.v2.1.patch, YARN-2575.v2.patch, > YARN-2575.v3.patch, YARN-2575.v4.patch, YARN-2575.v5.patch, > YARN-2575.v6.patch, YARN-2575.v7.patch, YARN-2575.v8.patch, YARN-2575.v9.patch > > > YARN-1051 introduces the ReservationSystem and in the current implementation > anyone who can submit applications can also submit reservations. This JIRA is > to evaluate creating separate ACLs for Reservation create/update/delete ops. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2575) Consider creating separate ACLs for Reservation create/update/delete/list ops
[ https://issues.apache.org/jira/browse/YARN-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15141920#comment-15141920 ] Sean Po commented on YARN-2575: --- Created a JIRA for documentation of Reservation ACLs. > Consider creating separate ACLs for Reservation create/update/delete/list ops > - > > Key: YARN-2575 > URL: https://issues.apache.org/jira/browse/YARN-2575 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, fairscheduler, resourcemanager >Reporter: Subru Krishnan >Assignee: Sean Po > Attachments: YARN-2575.v1.patch, YARN-2575.v10.patch, > YARN-2575.v11.patch, YARN-2575.v2.1.patch, YARN-2575.v2.patch, > YARN-2575.v3.patch, YARN-2575.v4.patch, YARN-2575.v5.patch, > YARN-2575.v6.patch, YARN-2575.v7.patch, YARN-2575.v8.patch, YARN-2575.v9.patch > > > YARN-1051 introduces the ReservationSystem and in the current implementation > anyone who can submit applications can also submit reservations. This JIRA is > to evaluate creating separate ACLs for Reservation create/update/delete ops. > Depends on YARN-4340 > Is related to YARN-4687 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2575) Consider creating separate ACLs for Reservation create/update/delete/list ops
[ https://issues.apache.org/jira/browse/YARN-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15141947#comment-15141947 ] Sean Po commented on YARN-2575: --- YARN-4687 > Consider creating separate ACLs for Reservation create/update/delete/list ops > - > > Key: YARN-2575 > URL: https://issues.apache.org/jira/browse/YARN-2575 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, fairscheduler, resourcemanager >Reporter: Subru Krishnan >Assignee: Sean Po > Attachments: YARN-2575.v1.patch, YARN-2575.v10.patch, > YARN-2575.v11.patch, YARN-2575.v2.1.patch, YARN-2575.v2.patch, > YARN-2575.v3.patch, YARN-2575.v4.patch, YARN-2575.v5.patch, > YARN-2575.v6.patch, YARN-2575.v7.patch, YARN-2575.v8.patch, YARN-2575.v9.patch > > > YARN-1051 introduces the ReservationSystem and in the current implementation > anyone who can submit applications can also submit reservations. This JIRA is > to evaluate creating separate ACLs for Reservation create/update/delete ops. > Depends on YARN-4340 > Is related to YARN-4687 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2575) Consider creating separate ACLs for Reservation create/update/delete/list ops
[ https://issues.apache.org/jira/browse/YARN-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15141948#comment-15141948 ] Hadoop QA commented on YARN-2575: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 17s {color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s {color} | {color:green} The patch appears to include 3 new or modified test files. {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 12s {color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 26s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 51s {color} | {color:green} trunk passed with JDK v1.8.0_72 {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 27s {color} | {color:green} trunk passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 41s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 47s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 39s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 43s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 52s {color} | {color:green} trunk passed with JDK v1.8.0_72 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m 6s {color} | {color:green} trunk passed with JDK v1.7.0_95 {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s {color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 31s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 47s {color} | {color:green} the patch passed with JDK v1.8.0_72 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 47s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 41s {color} | {color:green} the patch passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 41s {color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 45s {color} | {color:red} hadoop-yarn-project/hadoop-yarn: patch generated 3 new + 369 unchanged - 3 fixed = 372 total (was 372) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 55s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 45s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color} | {color:green} Patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 0s {color} | {color:green} The patch has no ill-formed XML file. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m 11s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 9s {color} | {color:green} the patch passed with JDK v1.8.0_72 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m 38s {color} | {color:green} the patch passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 36s {color} | {color:green} hadoop-yarn-api in the patch passed with JDK v1.8.0_72. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 43s {color} | {color:green} hadoop-yarn-common in the patch passed with JDK v1.8.0_72. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 79m 39s {color} | {color:red} hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_72. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 26s {color} | {color:green} hadoop-yarn-api in the patch passed with JDK v1.7.0_95. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 12s {color}
[jira] [Commented] (YARN-2575) Consider creating separate ACLs for Reservation create/update/delete/list ops
[ https://issues.apache.org/jira/browse/YARN-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15138761#comment-15138761 ] Hadoop QA commented on YARN-2575: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 14s {color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s {color} | {color:green} The patch appears to include 3 new or modified test files. {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s {color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m 35s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 48s {color} | {color:green} trunk passed with JDK v1.8.0_72 {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 6s {color} | {color:green} trunk passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 37s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 33s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 41s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 21s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 25s {color} | {color:green} trunk passed with JDK v1.8.0_72 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m 50s {color} | {color:green} trunk passed with JDK v1.7.0_95 {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s {color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 18s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 45s {color} | {color:green} the patch passed with JDK v1.8.0_72 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 45s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 5s {color} | {color:green} the patch passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 5s {color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 35s {color} | {color:red} hadoop-yarn-project/hadoop-yarn: patch generated 3 new + 370 unchanged - 3 fixed = 373 total (was 373) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 28s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 36s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color} | {color:green} Patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 0s {color} | {color:green} The patch has no ill-formed XML file. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 2s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 19s {color} | {color:green} the patch passed with JDK v1.8.0_72 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m 45s {color} | {color:green} the patch passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 20s {color} | {color:green} hadoop-yarn-api in the patch passed with JDK v1.8.0_72. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 55s {color} | {color:green} hadoop-yarn-common in the patch passed with JDK v1.8.0_72. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 65m 59s {color} | {color:red} hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_72. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 23s {color} | {color:green} hadoop-yarn-api in the patch passed with JDK v1.7.0_95. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 7s {color} |
[jira] [Commented] (YARN-2575) Consider creating separate ACLs for Reservation create/update/delete/list ops
[ https://issues.apache.org/jira/browse/YARN-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15139381#comment-15139381 ] Sean Po commented on YARN-2575: --- Re-uploading YARN-2575.v10.patch to fix unit test failures. > Consider creating separate ACLs for Reservation create/update/delete/list ops > - > > Key: YARN-2575 > URL: https://issues.apache.org/jira/browse/YARN-2575 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, fairscheduler, resourcemanager >Reporter: Subru Krishnan >Assignee: Sean Po > Attachments: YARN-2575.v1.patch, YARN-2575.v2.1.patch, > YARN-2575.v2.patch, YARN-2575.v3.patch, YARN-2575.v4.patch, > YARN-2575.v5.patch, YARN-2575.v6.patch, YARN-2575.v7.patch, > YARN-2575.v8.patch, YARN-2575.v9.patch > > > YARN-1051 introduces the ReservationSystem and in the current implementation > anyone who can submit applications can also submit reservations. This JIRA is > to evaluate creating separate ACLs for Reservation create/update/delete ops. > Depends on YARN-4340 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2575) Consider creating separate ACLs for Reservation create/update/delete/list ops
[ https://issues.apache.org/jira/browse/YARN-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15139964#comment-15139964 ] Subru Krishnan commented on YARN-2575: -- Thanks [~seanpo03] for the updated patch. It LGTM, just some feedback on *ClientRMService::checkReservationACLs*: * Let's use _ReservationId_ instead of _reservationCreatorName_ as the argument as the logic to determine the submitter is common * I feel it's better to separate the checks as the logic will be more explicit Can you run it in a single box and validate the patch. > Consider creating separate ACLs for Reservation create/update/delete/list ops > - > > Key: YARN-2575 > URL: https://issues.apache.org/jira/browse/YARN-2575 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, fairscheduler, resourcemanager >Reporter: Subru Krishnan >Assignee: Sean Po > Attachments: YARN-2575.v1.patch, YARN-2575.v10.patch, > YARN-2575.v2.1.patch, YARN-2575.v2.patch, YARN-2575.v3.patch, > YARN-2575.v4.patch, YARN-2575.v5.patch, YARN-2575.v6.patch, > YARN-2575.v7.patch, YARN-2575.v8.patch, YARN-2575.v9.patch > > > YARN-1051 introduces the ReservationSystem and in the current implementation > anyone who can submit applications can also submit reservations. This JIRA is > to evaluate creating separate ACLs for Reservation create/update/delete ops. > Depends on YARN-4340 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2575) Consider creating separate ACLs for Reservation create/update/delete/list ops
[ https://issues.apache.org/jira/browse/YARN-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15139954#comment-15139954 ] Hadoop QA commented on YARN-2575: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 21s {color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s {color} | {color:green} The patch appears to include 3 new or modified test files. {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s {color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m 11s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m 22s {color} | {color:green} trunk passed with JDK v1.8.0_72 {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m 28s {color} | {color:green} trunk passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 44s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m 13s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 53s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m 32s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 8s {color} | {color:green} trunk passed with JDK v1.8.0_72 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m 3s {color} | {color:green} trunk passed with JDK v1.7.0_95 {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s {color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 26s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 46s {color} | {color:green} the patch passed with JDK v1.8.0_72 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 46s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 19s {color} | {color:green} the patch passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 19s {color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 38s {color} | {color:red} hadoop-yarn-project/hadoop-yarn: patch generated 3 new + 369 unchanged - 3 fixed = 372 total (was 372) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 46s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 39s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color} | {color:green} Patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 1s {color} | {color:green} The patch has no ill-formed XML file. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 12s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 53s {color} | {color:green} the patch passed with JDK v1.8.0_72 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m 0s {color} | {color:green} the patch passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 33s {color} | {color:green} hadoop-yarn-api in the patch passed with JDK v1.8.0_72. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 35s {color} | {color:green} hadoop-yarn-common in the patch passed with JDK v1.8.0_72. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 72m 53s {color} | {color:red} hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_72. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 33s {color} | {color:green} hadoop-yarn-api in the patch passed with JDK v1.7.0_95. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 52s {color}
[jira] [Commented] (YARN-2575) Consider creating separate ACLs for Reservation create/update/delete/list ops
[ https://issues.apache.org/jira/browse/YARN-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15140473#comment-15140473 ] Arun Suresh commented on YARN-2575: --- Thank you for working on this [~seanpo03].. I took a the latest patch. It looks good overall and I like the coverage in the testcase. Few very Minor nit : # The {{ReservationsACLsManager}} manager has a no arg constructor which you've marked as *VisibileForTesting* but I dont see it being used. You can probably remove it # Typo in testcase method name. should be * createFairSchedulerConfiguration* # The Javadoc for *PlanView::getReservations* says that it returns a *ReservationAllocation* but you are actually returning a Set of ReservationAllocations +1 Pending the above, [~subru]'s outstanding comments, the last two checkstyle warnings, and a final run of Jenkins. > Consider creating separate ACLs for Reservation create/update/delete/list ops > - > > Key: YARN-2575 > URL: https://issues.apache.org/jira/browse/YARN-2575 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, fairscheduler, resourcemanager >Reporter: Subru Krishnan >Assignee: Sean Po > Attachments: YARN-2575.v1.patch, YARN-2575.v10.patch, > YARN-2575.v2.1.patch, YARN-2575.v2.patch, YARN-2575.v3.patch, > YARN-2575.v4.patch, YARN-2575.v5.patch, YARN-2575.v6.patch, > YARN-2575.v7.patch, YARN-2575.v8.patch, YARN-2575.v9.patch > > > YARN-1051 introduces the ReservationSystem and in the current implementation > anyone who can submit applications can also submit reservations. This JIRA is > to evaluate creating separate ACLs for Reservation create/update/delete ops. > Depends on YARN-4340 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2575) Consider creating separate ACLs for Reservation create/update/delete/list ops
[ https://issues.apache.org/jira/browse/YARN-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15138272#comment-15138272 ] Subru Krishnan commented on YARN-2575: -- Thanks [~seanpo03] for addressing my feedback. The latest patch looks very close, just a few minor comments: * I discussed offline with [~asuresh] and [~curino], for simplicity let's follow that user needs LIST_RESERVATIONS ACL for range list queries. This will simplify the code as we don't need special ACL handling for list queries. * With the above proposal, we can reorder *ClientRMService::checkReservationACLs* for better readability - first check if _ReservationACLManager is not null, then if the user is the owner, followed by whether he/she has the appropriate ACLs and finally if user is admin or not. * Nit: some small copy-paste typos still in Javadocs > Consider creating separate ACLs for Reservation create/update/delete/list ops > - > > Key: YARN-2575 > URL: https://issues.apache.org/jira/browse/YARN-2575 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, fairscheduler, resourcemanager >Reporter: Subru Krishnan >Assignee: Sean Po > Attachments: YARN-2575.v1.patch, YARN-2575.v2.1.patch, > YARN-2575.v2.patch, YARN-2575.v3.patch, YARN-2575.v4.patch, > YARN-2575.v5.patch, YARN-2575.v6.patch, YARN-2575.v7.patch, > YARN-2575.v8.patch, YARN-2575.v9.patch > > > YARN-1051 introduces the ReservationSystem and in the current implementation > anyone who can submit applications can also submit reservations. This JIRA is > to evaluate creating separate ACLs for Reservation create/update/delete ops. > Depends on YARN-4340 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2575) Consider creating separate ACLs for Reservation create/update/delete/list ops
[ https://issues.apache.org/jira/browse/YARN-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15134578#comment-15134578 ] Sean Po commented on YARN-2575: --- YARN-2575.v8.patch fixes a javadoc error that resulted from a typo in a javadoc. The remaining javadoc errors are caused by the use of '_' as an identifier which might not be supported in the future. They are not related to my changes. The checkstyle issues already existed in the past: Too many parameters: AllocationConfiguration.java:108: public AllocationConfiguration(MapminQueueResources,:10: More than 7 parameters (found 22). Method too long: AllocationFileLoaderService.java:205: public synchronized void reloadAllocations() throws IOException,:3: Method length is 216 lines (max allowed is 150). Too many parameters: AllocationFileLoaderService.java:427: private void loadQueue(String parentName, Element element,:16: More than 7 parameters (found 17). The test failures hadoop.yarn.server.resourcemanager.TestClientRMTokens and hadoop.yarn.server.resourcemanager.TestAMAuthorization are known failures and are covered in HADOOP-12687. > Consider creating separate ACLs for Reservation create/update/delete/list ops > - > > Key: YARN-2575 > URL: https://issues.apache.org/jira/browse/YARN-2575 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, fairscheduler, resourcemanager >Reporter: Subru Krishnan >Assignee: Sean Po > Attachments: YARN-2575.v1.patch, YARN-2575.v2.1.patch, > YARN-2575.v2.patch, YARN-2575.v3.patch, YARN-2575.v4.patch, > YARN-2575.v5.patch, YARN-2575.v6.patch, YARN-2575.v7.patch, YARN-2575.v8.patch > > > YARN-1051 introduces the ReservationSystem and in the current implementation > anyone who can submit applications can also submit reservations. This JIRA is > to evaluate creating separate ACLs for Reservation create/update/delete ops. > Depends on YARN-4340 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2575) Consider creating separate ACLs for Reservation create/update/delete/list ops
[ https://issues.apache.org/jira/browse/YARN-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15134958#comment-15134958 ] Hadoop QA commented on YARN-2575: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 11s {color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s {color} | {color:green} The patch appears to include 6 new or modified test files. {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 25s {color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m 53s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 56s {color} | {color:green} trunk passed with JDK v1.8.0_66 {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 5s {color} | {color:green} trunk passed with JDK v1.7.0_91 {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 38s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 32s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 38s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 27s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 33s {color} | {color:green} trunk passed with JDK v1.8.0_66 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m 56s {color} | {color:green} trunk passed with JDK v1.7.0_91 {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 24s {color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 19s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 53s {color} | {color:green} the patch passed with JDK v1.8.0_66 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 53s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 4s {color} | {color:green} the patch passed with JDK v1.7.0_91 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 4s {color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 37s {color} | {color:red} hadoop-yarn-project/hadoop-yarn: patch generated 3 new + 390 unchanged - 3 fixed = 393 total (was 393) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 29s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 33s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color} | {color:green} Patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 0s {color} | {color:green} The patch has no ill-formed XML file. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 53s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 24s {color} | {color:green} the patch passed with JDK v1.8.0_66 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m 51s {color} | {color:green} the patch passed with JDK v1.7.0_91 {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 23s {color} | {color:green} hadoop-yarn-api in the patch passed with JDK v1.8.0_66. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 56s {color} | {color:green} hadoop-yarn-common in the patch passed with JDK v1.8.0_66. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 62m 36s {color} | {color:red} hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 23s {color} | {color:green} hadoop-yarn-api in the patch passed with JDK v1.7.0_91. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 8s {color} |
[jira] [Commented] (YARN-2575) Consider creating separate ACLs for Reservation create/update/delete/list ops
[ https://issues.apache.org/jira/browse/YARN-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15133890#comment-15133890 ] Sean Po commented on YARN-2575: --- YARN-2575.v7.patch addresses the checkstyle, findbugs as well as: hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification and hadoop.yarn.conf.TestYarnConfigurationFields. hadoop.yarn.server.resourcemanager.TestClientRMTokens and hadoop.yarn.server.resourcemanager.TestAMAuthorization test failures are known failures and are covered in HADOOP-12687. > Consider creating separate ACLs for Reservation create/update/delete/list ops > - > > Key: YARN-2575 > URL: https://issues.apache.org/jira/browse/YARN-2575 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, fairscheduler, resourcemanager >Reporter: Subru Krishnan >Assignee: Sean Po > Attachments: YARN-2575.v1.patch, YARN-2575.v2.1.patch, > YARN-2575.v2.patch, YARN-2575.v3.patch, YARN-2575.v4.patch, > YARN-2575.v5.patch, YARN-2575.v6.patch, YARN-2575.v7.patch > > > YARN-1051 introduces the ReservationSystem and in the current implementation > anyone who can submit applications can also submit reservations. This JIRA is > to evaluate creating separate ACLs for Reservation create/update/delete ops. > Depends on YARN-4340 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2575) Consider creating separate ACLs for Reservation create/update/delete/list ops
[ https://issues.apache.org/jira/browse/YARN-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15134096#comment-15134096 ] Hadoop QA commented on YARN-2575: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 12s {color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 1s {color} | {color:green} The patch appears to include 6 new or modified test files. {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 26s {color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 13s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 8s {color} | {color:green} trunk passed with JDK v1.8.0_66 {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 17s {color} | {color:green} trunk passed with JDK v1.7.0_91 {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 39s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 41s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 41s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 40s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 37s {color} | {color:green} trunk passed with JDK v1.8.0_66 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m 11s {color} | {color:green} trunk passed with JDK v1.7.0_91 {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 26s {color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 24s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 4s {color} | {color:green} the patch passed with JDK v1.8.0_66 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 4s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 18s {color} | {color:green} the patch passed with JDK v1.7.0_91 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 18s {color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 37s {color} | {color:red} hadoop-yarn-project/hadoop-yarn: patch generated 3 new + 390 unchanged - 3 fixed = 393 total (was 393) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 35s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 36s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color} | {color:green} Patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 0s {color} | {color:green} The patch has no ill-formed XML file. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 21s {color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 22s {color} | {color:red} hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66. {color} | | {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 7m 27s {color} | {color:red} hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_91 with JDK v1.7.0_91 generated 1 new + 2 unchanged - 0 fixed = 3 total (was 2) {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m 1s {color} | {color:green} the patch passed with JDK v1.7.0_91 {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 24s {color} | {color:green} hadoop-yarn-api in the patch passed with JDK v1.8.0_66. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 2s {color} | {color:green} hadoop-yarn-common in the patch passed with JDK v1.8.0_66. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 64m 12s {color} | {color:red}
[jira] [Commented] (YARN-2575) Consider creating separate ACLs for Reservation create/update/delete/list ops
[ https://issues.apache.org/jira/browse/YARN-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15135380#comment-15135380 ] Sean Po commented on YARN-2575: --- Thanks [~subru] I took all your comments and put it into YARN-2575.v9.patch. > Consider creating separate ACLs for Reservation create/update/delete/list ops > - > > Key: YARN-2575 > URL: https://issues.apache.org/jira/browse/YARN-2575 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, fairscheduler, resourcemanager >Reporter: Subru Krishnan >Assignee: Sean Po > Attachments: YARN-2575.v1.patch, YARN-2575.v2.1.patch, > YARN-2575.v2.patch, YARN-2575.v3.patch, YARN-2575.v4.patch, > YARN-2575.v5.patch, YARN-2575.v6.patch, YARN-2575.v7.patch, > YARN-2575.v8.patch, YARN-2575.v9.patch > > > YARN-1051 introduces the ReservationSystem and in the current implementation > anyone who can submit applications can also submit reservations. This JIRA is > to evaluate creating separate ACLs for Reservation create/update/delete ops. > Depends on YARN-4340 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2575) Consider creating separate ACLs for Reservation create/update/delete/list ops
[ https://issues.apache.org/jira/browse/YARN-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15135312#comment-15135312 ] Subru Krishnan commented on YARN-2575: -- Thanks [~seanpo03] for submitting the patch. I reviewd it & please find my comments below: * We can remove *SchedulerUtils/AccessType* as the changes are redundant now * In the Javadocs of *ReservationACL*, make it explicit that whoever has created the reservations can update or delete it and administrator can modify & list all reservations * Rename CREATE_RESERVATIONS to SUBMIT_RESERVATIONS based on the above comment for *ReservationACL* * Rename _submitter_ to _reservationCreatorName_ in *ClientRMService* for better readability * Using a boolean array for conveying if the user has admin rights or not is complicating the code in *ClientRMService::checkReservationACLs*, replace it with a AtomicBoolean * In *ClientRMService::checkReservationACLs*, first check if ACLs are enabled or not, i.e. ACLsManager is null. If so, we can return immediately * Nit: since *ClientRMService::checkReservationACLs* is a private method, we can use empty as defaults and avoid the null checks to reduce the if/else checks * In *AbstractReservationSystem*, we should instantiate ACLsManager only if both YARN and reservations ACLs are enabled * There is a copy-paste typo in the Javdoc of *PlanView::getReservations* * In *CapacitySchedulerConfiguration*, _getReservationAcl/setAcl_ should be private. Also annotate test visibility for _setReservationAcls_ * Can we maintain consistency in the order of _operator_ and _orginalSubmitter_ in *ReservationACLsTestBase* as it's confusing now :) * In *ReservationACLsTestBase*, can we add the scenario for admin of queue should not be able to perform list operation on another queue * Some copy-paste typo in both invocations and Javadoc for _update_ test scenarios in *ReservationACLsTestBase* * Is it possible to parameterize the ReservationACLsTest instead of separate ones for Capacity/Fair scheduler? > Consider creating separate ACLs for Reservation create/update/delete/list ops > - > > Key: YARN-2575 > URL: https://issues.apache.org/jira/browse/YARN-2575 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, fairscheduler, resourcemanager >Reporter: Subru Krishnan >Assignee: Sean Po > Attachments: YARN-2575.v1.patch, YARN-2575.v2.1.patch, > YARN-2575.v2.patch, YARN-2575.v3.patch, YARN-2575.v4.patch, > YARN-2575.v5.patch, YARN-2575.v6.patch, YARN-2575.v7.patch, YARN-2575.v8.patch > > > YARN-1051 introduces the ReservationSystem and in the current implementation > anyone who can submit applications can also submit reservations. This JIRA is > to evaluate creating separate ACLs for Reservation create/update/delete ops. > Depends on YARN-4340 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2575) Consider creating separate ACLs for Reservation create/update/delete/list ops
[ https://issues.apache.org/jira/browse/YARN-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15135591#comment-15135591 ] Hadoop QA commented on YARN-2575: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 16s {color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s {color} | {color:green} The patch appears to include 3 new or modified test files. {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 34s {color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 42s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 47s {color} | {color:green} trunk passed with JDK v1.8.0_66 {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 21s {color} | {color:green} trunk passed with JDK v1.7.0_91 {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 38s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 42s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 40s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 38s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 52s {color} | {color:green} trunk passed with JDK v1.8.0_66 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m 1s {color} | {color:green} trunk passed with JDK v1.7.0_91 {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 25s {color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 27s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 45s {color} | {color:green} the patch passed with JDK v1.8.0_66 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 45s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 22s {color} | {color:green} the patch passed with JDK v1.7.0_91 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 22s {color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 38s {color} | {color:red} hadoop-yarn-project/hadoop-yarn: patch generated 3 new + 369 unchanged - 3 fixed = 372 total (was 372) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 37s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 36s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color} | {color:green} Patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 1s {color} | {color:green} The patch has no ill-formed XML file. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 19s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 51s {color} | {color:green} the patch passed with JDK v1.8.0_66 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m 0s {color} | {color:green} the patch passed with JDK v1.7.0_91 {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 32s {color} | {color:green} hadoop-yarn-api in the patch passed with JDK v1.8.0_66. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 20s {color} | {color:green} hadoop-yarn-common in the patch passed with JDK v1.8.0_66. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 69m 25s {color} | {color:red} hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 25s {color} | {color:green} hadoop-yarn-api in the patch passed with JDK v1.7.0_91. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 16s {color}
[jira] [Commented] (YARN-2575) Consider creating separate ACLs for Reservation create/update/delete/list ops
[ https://issues.apache.org/jira/browse/YARN-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15133730#comment-15133730 ] Hadoop QA commented on YARN-2575: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 25s {color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s {color} | {color:green} The patch appears to include 5 new or modified test files. {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 51s {color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 0s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 48s {color} | {color:green} trunk passed with JDK v1.8.0_72 {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 7s {color} | {color:green} trunk passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 40s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 40s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 44s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 31s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 27s {color} | {color:green} trunk passed with JDK v1.8.0_72 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m 58s {color} | {color:green} trunk passed with JDK v1.7.0_95 {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 26s {color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 19s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 43s {color} | {color:green} the patch passed with JDK v1.8.0_72 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 43s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 6s {color} | {color:green} the patch passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 6s {color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 37s {color} | {color:red} hadoop-yarn-project/hadoop-yarn: patch generated 11 new + 390 unchanged - 3 fixed = 401 total (was 393) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 32s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 36s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color} | {color:green} Patch has no whitespace issues. {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 18s {color} | {color:red} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager generated 5 new + 0 unchanged - 0 fixed = 5 total (was 0) {color} | | {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 20s {color} | {color:red} hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_72. {color} | | {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 7m 2s {color} | {color:red} hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95 with JDK v1.7.0_95 generated 1 new + 2 unchanged - 0 fixed = 3 total (was 2) {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m 49s {color} | {color:green} the patch passed with JDK v1.7.0_95 {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 20s {color} | {color:red} hadoop-yarn-api in the patch failed with JDK v1.8.0_72. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 55s {color} | {color:green} hadoop-yarn-common in the patch passed with JDK v1.8.0_72. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 67m 53s {color} | {color:red} hadoop-yarn-server-resourcemanager in the patch failed with
[jira] [Commented] (YARN-2575) Consider creating separate ACLs for Reservation create/update/delete/list ops
[ https://issues.apache.org/jira/browse/YARN-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15102876#comment-15102876 ] Subru Krishnan commented on YARN-2575: -- [~seanpo03], ReservationACLs does not ignore hierarchies as reservations are currently only supported on leaf queues. I hope that clarifies my suggestion. We should only look for ReservationACLs in the leaf queue in the Scheduler to keep it simple. Additionally we need to ensure an admin user can list/update/delete reservations. Unit test validating the same will be great. > Consider creating separate ACLs for Reservation create/update/delete/list ops > - > > Key: YARN-2575 > URL: https://issues.apache.org/jira/browse/YARN-2575 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, fairscheduler, resourcemanager >Reporter: Subru Krishnan >Assignee: Sean Po > Attachments: YARN-2575.v1.patch, YARN-2575.v2.1.patch, > YARN-2575.v2.patch, YARN-2575.v3.patch, YARN-2575.v4.patch > > > YARN-1051 introduces the ReservationSystem and in the current implementation > anyone who can submit applications can also submit reservations. This JIRA is > to evaluate creating separate ACLs for Reservation create/update/delete ops. > Depends on YARN-4340 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2575) Consider creating separate ACLs for Reservation create/update/delete/list ops
[ https://issues.apache.org/jira/browse/YARN-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15092467#comment-15092467 ] Sean Po commented on YARN-2575: --- >> When Reservation ACLs are not enabled - in this case, everyone should have >> access. This logic is already in place at Line 82 of ReservationsACLsManager. >> When Reservation ACLs are enabled but not defined - in this case also I >> think everyone should have access. Here are my thoughts on this: If we do implement it, the one problem that I see is that there might be inconsistencies in behaviour compared to QueueACLs checks because the queue hierarchy is ignored in ReservationACLs. For QueueACLs the behaviour is that if the queue is root and if ADMINISTER_QUEUE acl is not defined on root, then all users will have the ADMINISTER_QUEUE ACL on the root queue (Line 587 CapacitySchedulerConfiguration). For example: UserA has ADMINISTER_QUEUE ACL on queue root.QueueA, but not root.QueueB. If the queueACLs on root are not defined, UserA will also have ADMINISTER_QUEUE ACL on root.QueueB because QueueACLs checking process recursively looks up the hierarchy. With ReservationACLs, hierarchy is ignored, so if we made it so that everyone had ACLs on the queue if the ACL for the queue is not defined, then the UserA from the example above will have access to only root and root.QueueA. The good part about implementing this is that when someone is configuring YARN, they can get started a lot quicker without mucking around with the configuration. This will make it easier too if there is a team where everyone needs to make reservations on a particular queue, for example a test queue for team members to play around with. What are your thoughts [~subru] and [~asuresh]? > Consider creating separate ACLs for Reservation create/update/delete/list ops > - > > Key: YARN-2575 > URL: https://issues.apache.org/jira/browse/YARN-2575 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, fairscheduler, resourcemanager >Reporter: Subru Krishnan >Assignee: Sean Po > Attachments: YARN-2575.v1.patch, YARN-2575.v2.1.patch, > YARN-2575.v2.patch, YARN-2575.v3.patch, YARN-2575.v4.patch > > > YARN-1051 introduces the ReservationSystem and in the current implementation > anyone who can submit applications can also submit reservations. This JIRA is > to evaluate creating separate ACLs for Reservation create/update/delete ops. > Depends on YARN-4340 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2575) Consider creating separate ACLs for Reservation create/update/delete/list ops
[ https://issues.apache.org/jira/browse/YARN-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15081807#comment-15081807 ] Subru Krishnan commented on YARN-2575: -- I am wondering whether we have covered the behavior for the following 2 scenarios: * When Reservation ACLs are not enabled - in this case, everyone should have access. * When Reservation ACLs are enabled but not defined - in this case also I think everyone should have access. [~asuresh], can you confirm? > Consider creating separate ACLs for Reservation create/update/delete/list ops > - > > Key: YARN-2575 > URL: https://issues.apache.org/jira/browse/YARN-2575 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, fairscheduler, resourcemanager >Reporter: Subru Krishnan >Assignee: Sean Po > Attachments: YARN-2575.v1.patch, YARN-2575.v2.1.patch, > YARN-2575.v2.patch, YARN-2575.v3.patch, YARN-2575.v4.patch > > > YARN-1051 introduces the ReservationSystem and in the current implementation > anyone who can submit applications can also submit reservations. This JIRA is > to evaluate creating separate ACLs for Reservation create/update/delete ops. > Depends on YARN-4340 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2575) Consider creating separate ACLs for Reservation create/update/delete/list ops
[ https://issues.apache.org/jira/browse/YARN-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15074307#comment-15074307 ] Subru Krishnan commented on YARN-2575: -- Thanks @Sean Po for the updated patch. It looks mostly good to me, just a few minor comments: * Typo in Javadocs for *ReservationACL::LIST_RESERVATIONS* * Unrelated formatting changes in *ClientRMService* * In *ClientRMService::submitApplication*, handle null i.e. reservation id not being present in _Plan_ * _rmContext.getReservationSystem()_ is redudant as we already hold a reference to reservation system if enabled, so can be used directly * *YarnScheduler* should not be part of the patch * We should return the default ACLs, i.e. everyone has access in *ReservationSchedulerConfiguration::getReservationAcls* * Override tag missing in child classes for _getReservationAcls_ > Consider creating separate ACLs for Reservation create/update/delete/list ops > - > > Key: YARN-2575 > URL: https://issues.apache.org/jira/browse/YARN-2575 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, fairscheduler, resourcemanager >Reporter: Subru Krishnan >Assignee: Sean Po > Attachments: YARN-2575.v1.patch, YARN-2575.v2.1.patch, > YARN-2575.v2.patch, YARN-2575.v3.patch > > > YARN-1051 introduces the ReservationSystem and in the current implementation > anyone who can submit applications can also submit reservations. This JIRA is > to evaluate creating separate ACLs for Reservation create/update/delete ops. > Depends on YARN-4340 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2575) Consider creating separate ACLs for Reservation create/update/delete/list ops
[ https://issues.apache.org/jira/browse/YARN-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15063254#comment-15063254 ] Sean Po commented on YARN-2575: --- I'm working on simplifying the patch based on offline comments about consolidating the ReservationsACLsManager constructor using an abstract interface, and moving the ReservationsACLsManager into the AbstractReservationSystem. Also adding reservation checks in submit application if a reservatonID is specified. > Consider creating separate ACLs for Reservation create/update/delete/list ops > - > > Key: YARN-2575 > URL: https://issues.apache.org/jira/browse/YARN-2575 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, fairscheduler, resourcemanager >Reporter: Subru Krishnan >Assignee: Sean Po > Attachments: YARN-2575.v1.patch, YARN-2575.v2.1.patch, > YARN-2575.v2.patch > > > YARN-1051 introduces the ReservationSystem and in the current implementation > anyone who can submit applications can also submit reservations. This JIRA is > to evaluate creating separate ACLs for Reservation create/update/delete ops. > Depends on YARN-4340 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2575) Consider creating separate ACLs for Reservation create/update/delete/list ops
[ https://issues.apache.org/jira/browse/YARN-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060892#comment-15060892 ] Subru Krishnan commented on YARN-2575: -- Thanks [~seanpo03] for taking this up, this is an important patch. I took a look at your patch and have a few suggestions to simplify it: * Let us move the *ReservationACLs* to the *ReservationSystem* and add a getter. With this approach, we need not change the _RM_ or _ClientRMService_ constructor, a non-trivial chunk of the patch is addressing the resulting conflicts. * We should add an explicit ACL for *LIST_RESERVATIONS* to avoid the convoluted implied if checks. * I feel that we can replace the checks in the _Scheduler/Queue_ hierarchy with one in the *ClientRMService::submitApp*. * The changes in the scheduler xml files are not needed as the _ReservationSystem_ is not enabled by default yet. * In *ACLsTestBase*, you can refer to *TestReservationSystemWithRMHA::addNodeCapacityToPlan* for draining the dispatcher to ensure node is registered with RM. > Consider creating separate ACLs for Reservation create/update/delete/list ops > - > > Key: YARN-2575 > URL: https://issues.apache.org/jira/browse/YARN-2575 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, fairscheduler, resourcemanager >Reporter: Subru Krishnan >Assignee: Sean Po > Attachments: YARN-2575.v1.patch > > > YARN-1051 introduces the ReservationSystem and in the current implementation > anyone who can submit applications can also submit reservations. This JIRA is > to evaluate creating separate ACLs for Reservation create/update/delete ops. > Depends on YARN-4340 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2575) Consider creating separate ACLs for Reservation create/update/delete/list ops
[ https://issues.apache.org/jira/browse/YARN-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15059414#comment-15059414 ] Sean Po commented on YARN-2575: --- YARN-4340 is available for review. > Consider creating separate ACLs for Reservation create/update/delete/list ops > - > > Key: YARN-2575 > URL: https://issues.apache.org/jira/browse/YARN-2575 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler, fairscheduler, resourcemanager >Reporter: Subru Krishnan >Assignee: Sean Po > Attachments: YARN-2575.v1.patch > > > YARN-1051 introduces the ReservationSystem and in the current implementation > anyone who can submit applications can also submit reservations. This JIRA is > to evaluate creating separate ACLs for Reservation create/update/delete ops. > Depends on YARN-4340 -- This message was sent by Atlassian JIRA (v6.3.4#6332)