[jira] [Commented] (YARN-899) Get queue administration ACLs working

[Sandy Ryza (JIRA)]

[ 
https://issues.apache.org/jira/browse/YARN-899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13783324#comment-13783324
 ] 

Sandy Ryza commented on YARN-899:
-

It looks like this added a bunch of unused imports into TestFairScheduler.  
Filed YARN-1263 for getting rid of these.

> Get queue administration ACLs working
> -
>
> Key: YARN-899
> URL: https://issues.apache.org/jira/browse/YARN-899
> Project: Hadoop YARN
>  Issue Type: Sub-task
>  Components: scheduler
>Affects Versions: 2.1.0-beta
>Reporter: Sandy Ryza
>Assignee: Xuan Gong
> Fix For: 2.1.2-beta
>
> Attachments: YARN-899.10.patch, YARN-899.10.patch, YARN-899.11.patch, 
> YARN-899.12.patch, YARN-899.1.patch, YARN-899.2.patch, YARN-899.3.patch, 
> YARN-899.4.patch, YARN-899.5.patch, YARN-899.5.patch, YARN-899.6.patch, 
> YARN-899.7.patch, YARN-899.8.patch, YARN-899.9.patch
>
>
> The Capacity Scheduler documents the 
> yarn.scheduler.capacity.root..acl_administer_queue config option 
> for controlling who can administer a queue, but it is not hooked up to 
> anything.  The Fair Scheduler could make use of a similar option as well.  
> This is a feature-parity regression from MR1.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (YARN-899) Get queue administration ACLs working

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/YARN-899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13781384#comment-13781384
 ] 

Hudson commented on YARN-899:
-

FAILURE: Integrated in Hadoop-Mapreduce-trunk #1563 (See 
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1563/])
YARN-899. Added back queue level administrator-acls so that there is no 
regression w.r.t 1.x. Contributed by Xuan Gong. (vinodkv: 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1527282)
* 
/hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/ResourceSchedulerWrapper.java
* /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/Queue.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/YarnScheduler.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/security/QueueACLsManager.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/AppBlock.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebApp.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationACLs.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestClientRMService.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestClientRMTokens.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestQueueACLs.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMRestart.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestLeafQueue.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestClientToAMTokens.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/te

[jira] [Commented] (YARN-899) Get queue administration ACLs working

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/YARN-899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13781369#comment-13781369
 ] 

Hudson commented on YARN-899:
-

SUCCESS: Integrated in Hadoop-Hdfs-trunk #1537 (See 
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/1537/])
YARN-899. Added back queue level administrator-acls so that there is no 
regression w.r.t 1.x. Contributed by Xuan Gong. (vinodkv: 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1527282)
* 
/hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/ResourceSchedulerWrapper.java
* /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/Queue.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/YarnScheduler.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/security/QueueACLsManager.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/AppBlock.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebApp.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationACLs.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestClientRMService.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestClientRMTokens.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestQueueACLs.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMRestart.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestLeafQueue.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestClientToAMTokens.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/or

[jira] [Commented] (YARN-899) Get queue administration ACLs working

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/YARN-899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13781340#comment-13781340
 ] 

Hudson commented on YARN-899:
-

FAILURE: Integrated in Hadoop-Yarn-trunk #347 (See 
[https://builds.apache.org/job/Hadoop-Yarn-trunk/347/])
YARN-899. Added back queue level administrator-acls so that there is no 
regression w.r.t 1.x. Contributed by Xuan Gong. (vinodkv: 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1527282)
* 
/hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/ResourceSchedulerWrapper.java
* /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/Queue.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/YarnScheduler.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/security/QueueACLsManager.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/AppBlock.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebApp.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationACLs.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestClientRMService.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestClientRMTokens.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestQueueACLs.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMRestart.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestLeafQueue.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestClientToAMTokens.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/

[jira] [Commented] (YARN-899) Get queue administration ACLs working

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/YARN-899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13781308#comment-13781308
 ] 

Hudson commented on YARN-899:
-

SUCCESS: Integrated in Hadoop-trunk-Commit #4495 (See 
[https://builds.apache.org/job/Hadoop-trunk-Commit/4495/])
YARN-899. Added back queue level administrator-acls so that there is no 
regression w.r.t 1.x. Contributed by Xuan Gong. (vinodkv: 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1527282)
* 
/hadoop/common/trunk/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/ResourceSchedulerWrapper.java
* /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/Queue.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/YarnScheduler.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/security/QueueACLsManager.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/AppBlock.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebApp.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationACLs.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestClientRMService.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestClientRMTokens.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestQueueACLs.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMRestart.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestLeafQueue.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestClientToAMTokens.java
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/jav

[jira] [Commented] (YARN-899) Get queue administration ACLs working

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/YARN-899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13781299#comment-13781299
 ] 

Hadoop QA commented on YARN-899:


{color:green}+1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12605774/YARN-899.12.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 14 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 eclipse:eclipse{color}.  The patch built with 
eclipse:eclipse.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in 
hadoop-tools/hadoop-sls 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.

{color:green}+1 contrib tests{color}.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-YARN-Build/2035//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/2035//console

This message is automatically generated.

> Get queue administration ACLs working
> -
>
> Key: YARN-899
> URL: https://issues.apache.org/jira/browse/YARN-899
> Project: Hadoop YARN
>  Issue Type: Sub-task
>  Components: scheduler
>Affects Versions: 2.1.0-beta
>Reporter: Sandy Ryza
>Assignee: Xuan Gong
> Attachments: YARN-899.10.patch, YARN-899.10.patch, YARN-899.11.patch, 
> YARN-899.12.patch, YARN-899.1.patch, YARN-899.2.patch, YARN-899.3.patch, 
> YARN-899.4.patch, YARN-899.5.patch, YARN-899.5.patch, YARN-899.6.patch, 
> YARN-899.7.patch, YARN-899.8.patch, YARN-899.9.patch
>
>
> The Capacity Scheduler documents the 
> yarn.scheduler.capacity.root..acl_administer_queue config option 
> for controlling who can administer a queue, but it is not hooked up to 
> anything.  The Fair Scheduler could make use of a similar option as well.  
> This is a feature-parity regression from MR1.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (YARN-899) Get queue administration ACLs working

[Vinod Kumar Vavilapalli (JIRA)]

[ 
https://issues.apache.org/jira/browse/YARN-899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13781298#comment-13781298
 ] 

Vinod Kumar Vavilapalli commented on YARN-899:
--

Filed MAPREDUCE-5548, YARN-1248, YARN-1249 and YARN-1250 for the pending work.

> Get queue administration ACLs working
> -
>
> Key: YARN-899
> URL: https://issues.apache.org/jira/browse/YARN-899
> Project: Hadoop YARN
>  Issue Type: Sub-task
>  Components: scheduler
>Affects Versions: 2.1.0-beta
>Reporter: Sandy Ryza
>Assignee: Xuan Gong
> Attachments: YARN-899.10.patch, YARN-899.10.patch, YARN-899.11.patch, 
> YARN-899.12.patch, YARN-899.1.patch, YARN-899.2.patch, YARN-899.3.patch, 
> YARN-899.4.patch, YARN-899.5.patch, YARN-899.5.patch, YARN-899.6.patch, 
> YARN-899.7.patch, YARN-899.8.patch, YARN-899.9.patch
>
>
> The Capacity Scheduler documents the 
> yarn.scheduler.capacity.root..acl_administer_queue config option 
> for controlling who can administer a queue, but it is not hooked up to 
> anything.  The Fair Scheduler could make use of a similar option as well.  
> This is a feature-parity regression from MR1.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (YARN-899) Get queue administration ACLs working

[Vinod Kumar Vavilapalli (JIRA)]

[ 
https://issues.apache.org/jira/browse/YARN-899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13781293#comment-13781293
 ] 

Vinod Kumar Vavilapalli commented on YARN-899:
--

bq. One other thing I noticed is that queue-acls as they are in the patch don't 
work well with hierarchical queues. In the interest of removing the regression 
from 1.0, I'll push this and track queue hierarchy separately.
Actually, they look like they already work. A queue-admin at any level queue 
can submit to and kill apps in any sub-queue. We just need to write tests.

> Get queue administration ACLs working
> -
>
> Key: YARN-899
> URL: https://issues.apache.org/jira/browse/YARN-899
> Project: Hadoop YARN
>  Issue Type: Sub-task
>  Components: scheduler
>Affects Versions: 2.1.0-beta
>Reporter: Sandy Ryza
>Assignee: Xuan Gong
> Attachments: YARN-899.10.patch, YARN-899.10.patch, YARN-899.11.patch, 
> YARN-899.12.patch, YARN-899.1.patch, YARN-899.2.patch, YARN-899.3.patch, 
> YARN-899.4.patch, YARN-899.5.patch, YARN-899.5.patch, YARN-899.6.patch, 
> YARN-899.7.patch, YARN-899.8.patch, YARN-899.9.patch
>
>
> The Capacity Scheduler documents the 
> yarn.scheduler.capacity.root..acl_administer_queue config option 
> for controlling who can administer a queue, but it is not hooked up to 
> anything.  The Fair Scheduler could make use of a similar option as well.  
> This is a feature-parity regression from MR1.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (YARN-899) Get queue administration ACLs working

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/YARN-899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13781217#comment-13781217
 ] 

Hadoop QA commented on YARN-899:


{color:green}+1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12605763/YARN-899.11.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 13 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 eclipse:eclipse{color}.  The patch built with 
eclipse:eclipse.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in 
hadoop-tools/hadoop-sls 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.

{color:green}+1 contrib tests{color}.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-YARN-Build/2034//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/2034//console

This message is automatically generated.

> Get queue administration ACLs working
> -
>
> Key: YARN-899
> URL: https://issues.apache.org/jira/browse/YARN-899
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: scheduler
>Affects Versions: 2.1.0-beta
>Reporter: Sandy Ryza
>Assignee: Xuan Gong
> Attachments: YARN-899.10.patch, YARN-899.10.patch, YARN-899.11.patch, 
> YARN-899.1.patch, YARN-899.2.patch, YARN-899.3.patch, YARN-899.4.patch, 
> YARN-899.5.patch, YARN-899.5.patch, YARN-899.6.patch, YARN-899.7.patch, 
> YARN-899.8.patch, YARN-899.9.patch
>
>
> The Capacity Scheduler documents the 
> yarn.scheduler.capacity.root..acl_administer_queue config option 
> for controlling who can administer a queue, but it is not hooked up to 
> anything.  The Fair Scheduler could make use of a similar option as well.  
> This is a feature-parity regression from MR1.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (YARN-899) Get queue administration ACLs working

[Xuan Gong (JIRA)]

[ 
https://issues.apache.org/jira/browse/YARN-899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13780716#comment-13780716
 ] 

Xuan Gong commented on YARN-899:


Not sure the compiler error is because this:
{code}
[ERROR] 
/Users/xuan/2/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/ResourceSchedulerWrapper.java:[97,7]
 org.apache.hadoop.yarn.sls.scheduler.ResourceSchedulerWrapper is not abstract 
and does not override abstract method 
checkAccess(org.apache.hadoop.security.UserGroupInformation,org.apache.hadoop.yarn.api.records.QueueACL,java.lang.String)
 in org.apache.hadoop.yarn.server.resourcemanager.scheduler.ResourceScheduler
{code}

> Get queue administration ACLs working
> -
>
> Key: YARN-899
> URL: https://issues.apache.org/jira/browse/YARN-899
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: scheduler
>Affects Versions: 2.1.0-beta
>Reporter: Sandy Ryza
>Assignee: Xuan Gong
> Attachments: YARN-899.10.patch, YARN-899.10.patch, YARN-899.1.patch, 
> YARN-899.2.patch, YARN-899.3.patch, YARN-899.4.patch, YARN-899.5.patch, 
> YARN-899.5.patch, YARN-899.6.patch, YARN-899.7.patch, YARN-899.8.patch, 
> YARN-899.9.patch
>
>
> The Capacity Scheduler documents the 
> yarn.scheduler.capacity.root..acl_administer_queue config option 
> for controlling who can administer a queue, but it is not hooked up to 
> anything.  The Fair Scheduler could make use of a similar option as well.  
> This is a feature-parity regression from MR1.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (YARN-899) Get queue administration ACLs working

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/YARN-899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13780685#comment-13780685
 ] 

Hadoop QA commented on YARN-899:


{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12605634/YARN-899.10.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 13 new 
or modified test files.

{color:red}-1 javac{color:red}.  The patch appears to cause the build to 
fail.

Console output: https://builds.apache.org/job/PreCommit-YARN-Build/2033//console

This message is automatically generated.

> Get queue administration ACLs working
> -
>
> Key: YARN-899
> URL: https://issues.apache.org/jira/browse/YARN-899
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: scheduler
>Affects Versions: 2.1.0-beta
>Reporter: Sandy Ryza
>Assignee: Xuan Gong
> Attachments: YARN-899.10.patch, YARN-899.10.patch, YARN-899.1.patch, 
> YARN-899.2.patch, YARN-899.3.patch, YARN-899.4.patch, YARN-899.5.patch, 
> YARN-899.5.patch, YARN-899.6.patch, YARN-899.7.patch, YARN-899.8.patch, 
> YARN-899.9.patch
>
>
> The Capacity Scheduler documents the 
> yarn.scheduler.capacity.root..acl_administer_queue config option 
> for controlling who can administer a queue, but it is not hooked up to 
> anything.  The Fair Scheduler could make use of a similar option as well.  
> This is a feature-parity regression from MR1.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (YARN-899) Get queue administration ACLs working

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/YARN-899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779182#comment-13779182
 ] 

Hadoop QA commented on YARN-899:


{color:green}+1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12605320/YARN-899.9.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 11 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 eclipse:eclipse{color}.  The patch built with 
eclipse:eclipse.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.

{color:green}+1 contrib tests{color}.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-YARN-Build/2024//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/2024//console

This message is automatically generated.

> Get queue administration ACLs working
> -
>
> Key: YARN-899
> URL: https://issues.apache.org/jira/browse/YARN-899
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: scheduler
>Affects Versions: 2.1.0-beta
>Reporter: Sandy Ryza
>Assignee: Xuan Gong
> Attachments: YARN-899.1.patch, YARN-899.2.patch, YARN-899.3.patch, 
> YARN-899.4.patch, YARN-899.5.patch, YARN-899.5.patch, YARN-899.6.patch, 
> YARN-899.7.patch, YARN-899.8.patch, YARN-899.9.patch
>
>
> The Capacity Scheduler documents the 
> yarn.scheduler.capacity.root..acl_administer_queue config option 
> for controlling who can administer a queue, but it is not hooked up to 
> anything.  The Fair Scheduler could make use of a similar option as well.  
> This is a feature-parity regression from MR1.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (YARN-899) Get queue administration ACLs working

[Zhijie Shen (JIRA)]

[ 
https://issues.apache.org/jira/browse/YARN-899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779074#comment-13779074
 ] 

Zhijie Shen commented on YARN-899:
--

bq. Someone more experienced correct me if I'm wrong here, but I believe the 
goal of queue administration ACLs is to allow admins to delegate 
responsibility. So if I am a cluster admin and I set up a queue for the 
marketing department and a queue for the engineering department, I might want 
to allow the head of marketing to kill applications in the marketing queue 
without needing to go through me. With this in mind, I think who has access 
should be based on a union of ACLs - I should be able to kill any application 
in the marketing queue either if I am on the application's ACL or if I am on 
the queue's ACL.

Make sense

bq. For the view access, we can check the union of QueueACLs and ApplicationACLs

Then, IMHO, the union of ACLs should be applied to both viewing applications 
and killing them. More, I think it's good to document the super permission from 
the queue administrator.

> Get queue administration ACLs working
> -
>
> Key: YARN-899
> URL: https://issues.apache.org/jira/browse/YARN-899
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: scheduler
>Affects Versions: 2.1.0-beta
>Reporter: Sandy Ryza
>Assignee: Xuan Gong
> Attachments: YARN-899.1.patch, YARN-899.2.patch, YARN-899.3.patch, 
> YARN-899.4.patch, YARN-899.5.patch, YARN-899.5.patch, YARN-899.6.patch, 
> YARN-899.7.patch, YARN-899.8.patch
>
>
> The Capacity Scheduler documents the 
> yarn.scheduler.capacity.root..acl_administer_queue config option 
> for controlling who can administer a queue, but it is not hooked up to 
> anything.  The Fair Scheduler could make use of a similar option as well.  
> This is a feature-parity regression from MR1.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (YARN-899) Get queue administration ACLs working

[Xuan Gong (JIRA)]

[ 
https://issues.apache.org/jira/browse/YARN-899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13778001#comment-13778001
 ] 

Xuan Gong commented on YARN-899:


bq.Is the following code necessary?

Will remove

bq.Why not defining checkAccess in YarnScheduler? I saw other ACL related 
method is defined there as well.

Will move to YarnScheduler

bq.checkViewAccess is used in getApplications and getApplicationReport, does 
"||" mean the user doesn't need to have the access to the application whenever 
it has the access to the queue?

For the view access, we can check the union of QueueACLs and ApplicationACLs

> Get queue administration ACLs working
> -
>
> Key: YARN-899
> URL: https://issues.apache.org/jira/browse/YARN-899
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: scheduler
>Affects Versions: 2.1.0-beta
>Reporter: Sandy Ryza
>Assignee: Xuan Gong
> Attachments: YARN-899.1.patch, YARN-899.2.patch, YARN-899.3.patch, 
> YARN-899.4.patch, YARN-899.5.patch, YARN-899.5.patch, YARN-899.6.patch, 
> YARN-899.7.patch, YARN-899.8.patch
>
>
> The Capacity Scheduler documents the 
> yarn.scheduler.capacity.root..acl_administer_queue config option 
> for controlling who can administer a queue, but it is not hooked up to 
> anything.  The Fair Scheduler could make use of a similar option as well.  
> This is a feature-parity regression from MR1.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (YARN-899) Get queue administration ACLs working

[Zhijie Shen (JIRA)]

[ 
https://issues.apache.org/jira/browse/YARN-899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1399#comment-1399
 ] 

Zhijie Shen commented on YARN-899:
--

1. Is the following code necessary?
{code}
+System.out.println(callerUGI);
{code}

2. Why not defining checkAccess in YarnScheduler? I saw other ACL related 
method is defined there as well.
{code}
+  boolean checkAccess(UserGroupInformation callerUGI,
+  QueueACL acl, String queueName);
{code}

3. checkViewAccess is used in getApplications and getApplicationReport, does 
"||" mean the user doesn't need to have the access to the application whenever 
it has the access to the queue?
{code}
-  private boolean checkAccess(UserGroupInformation callerUGI, String owner,
-  ApplicationAccessType operationPerformed, ApplicationId applicationId) {
+  private boolean checkViewAccess(UserGroupInformation callerUGI, String owner,
+  ApplicationAccessType operationPerformed,
+  RMApp application) {
 return applicationsACLsManager.checkAccess(callerUGI, operationPerformed,
-owner, applicationId);
+owner, application.getApplicationId())
+|| queueACLsManager.checkAccess(callerUGI, QueueACL.ADMINISTER_QUEUE,
+application.getQueue());
{code}

> Get queue administration ACLs working
> -
>
> Key: YARN-899
> URL: https://issues.apache.org/jira/browse/YARN-899
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: scheduler
>Affects Versions: 2.1.0-beta
>Reporter: Sandy Ryza
>Assignee: Xuan Gong
> Attachments: YARN-899.1.patch, YARN-899.2.patch, YARN-899.3.patch, 
> YARN-899.4.patch, YARN-899.5.patch, YARN-899.5.patch, YARN-899.6.patch, 
> YARN-899.7.patch, YARN-899.8.patch
>
>
> The Capacity Scheduler documents the 
> yarn.scheduler.capacity.root..acl_administer_queue config option 
> for controlling who can administer a queue, but it is not hooked up to 
> anything.  The Fair Scheduler could make use of a similar option as well.  
> This is a feature-parity regression from MR1.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (YARN-899) Get queue administration ACLs working

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/YARN-899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13777069#comment-13777069
 ] 

Hadoop QA commented on YARN-899:


{color:green}+1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12604938/YARN-899.8.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 11 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 eclipse:eclipse{color}.  The patch built with 
eclipse:eclipse.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.

{color:green}+1 contrib tests{color}.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-YARN-Build/2010//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/2010//console

This message is automatically generated.

> Get queue administration ACLs working
> -
>
> Key: YARN-899
> URL: https://issues.apache.org/jira/browse/YARN-899
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: scheduler
>Affects Versions: 2.1.0-beta
>Reporter: Sandy Ryza
>Assignee: Xuan Gong
> Attachments: YARN-899.1.patch, YARN-899.2.patch, YARN-899.3.patch, 
> YARN-899.4.patch, YARN-899.5.patch, YARN-899.5.patch, YARN-899.6.patch, 
> YARN-899.7.patch, YARN-899.8.patch
>
>
> The Capacity Scheduler documents the 
> yarn.scheduler.capacity.root..acl_administer_queue config option 
> for controlling who can administer a queue, but it is not hooked up to 
> anything.  The Fair Scheduler could make use of a similar option as well.  
> This is a feature-parity regression from MR1.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (YARN-899) Get queue administration ACLs working

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/YARN-899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13749569#comment-13749569
 ] 

Hadoop QA commented on YARN-899:


{color:green}+1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12599465/YARN-899.6.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 10 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 eclipse:eclipse{color}.  The patch built with 
eclipse:eclipse.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.

{color:green}+1 contrib tests{color}.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-YARN-Build/1763//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1763//console

This message is automatically generated.

> Get queue administration ACLs working
> -
>
> Key: YARN-899
> URL: https://issues.apache.org/jira/browse/YARN-899
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: scheduler
>Affects Versions: 2.1.0-beta
>Reporter: Sandy Ryza
>Assignee: Xuan Gong
> Attachments: YARN-899.1.patch, YARN-899.2.patch, YARN-899.3.patch, 
> YARN-899.4.patch, YARN-899.5.patch, YARN-899.5.patch, YARN-899.6.patch
>
>
> The Capacity Scheduler documents the 
> yarn.scheduler.capacity.root..acl_administer_queue config option 
> for controlling who can administer a queue, but it is not hooked up to 
> anything.  The Fair Scheduler could make use of a similar option as well.  
> This is a feature-parity regression from MR1.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (YARN-899) Get queue administration ACLs working

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/YARN-899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13736527#comment-13736527
 ] 

Hadoop QA commented on YARN-899:


{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12597397/YARN-899.5.patch
  against trunk revision .

{color:red}-1 patch{color}.  Trunk compilation may be broken.

Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1693//console

This message is automatically generated.

> Get queue administration ACLs working
> -
>
> Key: YARN-899
> URL: https://issues.apache.org/jira/browse/YARN-899
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: scheduler
>Affects Versions: 2.1.0-beta
>Reporter: Sandy Ryza
>Assignee: Xuan Gong
> Attachments: YARN-899.1.patch, YARN-899.2.patch, YARN-899.3.patch, 
> YARN-899.4.patch, YARN-899.5.patch, YARN-899.5.patch
>
>
> The Capacity Scheduler documents the 
> yarn.scheduler.capacity.root..acl_administer_queue config option 
> for controlling who can administer a queue, but it is not hooked up to 
> anything.  The Fair Scheduler could make use of a similar option as well.  
> This is a feature-parity regression from MR1.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (YARN-899) Get queue administration ACLs working

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/YARN-899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13735956#comment-13735956
 ] 

Hadoop QA commented on YARN-899:


{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12597288/YARN-899.5.patch
  against trunk revision .

{color:red}-1 patch{color}.  Trunk compilation may be broken.

Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1690//console

This message is automatically generated.

> Get queue administration ACLs working
> -
>
> Key: YARN-899
> URL: https://issues.apache.org/jira/browse/YARN-899
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: scheduler
>Affects Versions: 2.1.0-beta
>Reporter: Sandy Ryza
>Assignee: Xuan Gong
> Attachments: YARN-899.1.patch, YARN-899.2.patch, YARN-899.3.patch, 
> YARN-899.4.patch, YARN-899.5.patch
>
>
> The Capacity Scheduler documents the 
> yarn.scheduler.capacity.root..acl_administer_queue config option 
> for controlling who can administer a queue, but it is not hooked up to 
> anything.  The Fair Scheduler could make use of a similar option as well.  
> This is a feature-parity regression from MR1.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (YARN-899) Get queue administration ACLs working

[Xuan Gong (JIRA)]

[ 
https://issues.apache.org/jira/browse/YARN-899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13735954#comment-13735954
 ] 

Xuan Gong commented on YARN-899:


Thanks for the comments. Sandy. New patch addresses all the comments

> Get queue administration ACLs working
> -
>
> Key: YARN-899
> URL: https://issues.apache.org/jira/browse/YARN-899
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: scheduler
>Affects Versions: 2.1.0-beta
>Reporter: Sandy Ryza
>Assignee: Xuan Gong
> Attachments: YARN-899.1.patch, YARN-899.2.patch, YARN-899.3.patch, 
> YARN-899.4.patch, YARN-899.5.patch
>
>
> The Capacity Scheduler documents the 
> yarn.scheduler.capacity.root..acl_administer_queue config option 
> for controlling who can administer a queue, but it is not hooked up to 
> anything.  The Fair Scheduler could make use of a similar option as well.  
> This is a feature-parity regression from MR1.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (YARN-899) Get queue administration ACLs working

[Sandy Ryza (JIRA)]

[ 
https://issues.apache.org/jira/browse/YARN-899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13735605#comment-13735605
 ] 

Sandy Ryza commented on YARN-899:
-

Oh, makes sense because the NodeManager uses it as well.  In that case the 
current approach is fine.

> Get queue administration ACLs working
> -
>
> Key: YARN-899
> URL: https://issues.apache.org/jira/browse/YARN-899
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: scheduler
>Affects Versions: 2.1.0-beta
>Reporter: Sandy Ryza
>Assignee: Xuan Gong
> Attachments: YARN-899.1.patch, YARN-899.2.patch, YARN-899.3.patch, 
> YARN-899.4.patch
>
>
> The Capacity Scheduler documents the 
> yarn.scheduler.capacity.root..acl_administer_queue config option 
> for controlling who can administer a queue, but it is not hooked up to 
> anything.  The Fair Scheduler could make use of a similar option as well.  
> This is a feature-parity regression from MR1.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (YARN-899) Get queue administration ACLs working

[Xuan Gong (JIRA)]

[ 
https://issues.apache.org/jira/browse/YARN-899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13735564#comment-13735564
 ] 

Xuan Gong commented on YARN-899:


bq.Do we need a QueueACLsManager separate from ApplicationACLsManager? I.e. 
could we just have a reference to ResourceScheduler in ApplicationACLsManager 
and check access based on the queue name there?

ResourceScheduler is in hadoop.yarn.server.resourcemanager package, but 
ApplicationACLsManager is in hadoop.yarn.common package, and looks like we can 
not add ResourceScheduler reference into ApplicationACLsManager.

> Get queue administration ACLs working
> -
>
> Key: YARN-899
> URL: https://issues.apache.org/jira/browse/YARN-899
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: scheduler
>Affects Versions: 2.1.0-beta
>Reporter: Sandy Ryza
>Assignee: Xuan Gong
> Attachments: YARN-899.1.patch, YARN-899.2.patch, YARN-899.3.patch, 
> YARN-899.4.patch
>
>
> The Capacity Scheduler documents the 
> yarn.scheduler.capacity.root..acl_administer_queue config option 
> for controlling who can administer a queue, but it is not hooked up to 
> anything.  The Fair Scheduler could make use of a similar option as well.  
> This is a feature-parity regression from MR1.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (YARN-899) Get queue administration ACLs working

[Sandy Ryza (JIRA)]

[ 
https://issues.apache.org/jira/browse/YARN-899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13735358#comment-13735358
 ] 

Sandy Ryza commented on YARN-899:
-

The general approach and patch are looking good.  Some comments:

Do we need a QueueACLsManager separate from ApplicationACLsManager?  I.e. could 
we just have a reference to ResourceScheduler in ApplicationACLsManager and 
check access based on the queue name there?

The LOG.debugs in the scheduler checkAccess methods should be surrounded by 
isDebugEnabled()

Filed YARN-1052 for moving submit ACLs outside the scheduler. 

> Get queue administration ACLs working
> -
>
> Key: YARN-899
> URL: https://issues.apache.org/jira/browse/YARN-899
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: scheduler
>Affects Versions: 2.1.0-beta
>Reporter: Sandy Ryza
>Assignee: Xuan Gong
> Attachments: YARN-899.1.patch, YARN-899.2.patch, YARN-899.3.patch, 
> YARN-899.4.patch
>
>
> The Capacity Scheduler documents the 
> yarn.scheduler.capacity.root..acl_administer_queue config option 
> for controlling who can administer a queue, but it is not hooked up to 
> anything.  The Fair Scheduler could make use of a similar option as well.  
> This is a feature-parity regression from MR1.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (YARN-899) Get queue administration ACLs working

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/YARN-899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13735204#comment-13735204
 ] 

Hadoop QA commented on YARN-899:


{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12597148/YARN-899.4.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 10 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:red}-1 javadoc{color}.  The javadoc tool appears to have generated 1 
warning messages.

{color:green}+1 eclipse:eclipse{color}.  The patch built with 
eclipse:eclipse.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.

{color:green}+1 contrib tests{color}.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-YARN-Build/1684//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1684//console

This message is automatically generated.

> Get queue administration ACLs working
> -
>
> Key: YARN-899
> URL: https://issues.apache.org/jira/browse/YARN-899
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: scheduler
>Affects Versions: 2.1.0-beta
>Reporter: Sandy Ryza
>Assignee: Xuan Gong
> Attachments: YARN-899.1.patch, YARN-899.2.patch, YARN-899.3.patch, 
> YARN-899.4.patch
>
>
> The Capacity Scheduler documents the 
> yarn.scheduler.capacity.root..acl_administer_queue config option 
> for controlling who can administer a queue, but it is not hooked up to 
> anything.  The Fair Scheduler could make use of a similar option as well.  
> This is a feature-parity regression from MR1.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (YARN-899) Get queue administration ACLs working

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/YARN-899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13735126#comment-13735126
 ] 

Hadoop QA commented on YARN-899:


{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12597140/YARN-899.3.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 20 new 
or modified test files.

{color:red}-1 patch{color}.  The patch command could not apply the patch.

Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1683//console

This message is automatically generated.

> Get queue administration ACLs working
> -
>
> Key: YARN-899
> URL: https://issues.apache.org/jira/browse/YARN-899
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: scheduler
>Affects Versions: 2.1.0-beta
>Reporter: Sandy Ryza
>Assignee: Xuan Gong
> Attachments: YARN-899.1.patch, YARN-899.2.patch, YARN-899.3.patch
>
>
> The Capacity Scheduler documents the 
> yarn.scheduler.capacity.root..acl_administer_queue config option 
> for controlling who can administer a queue, but it is not hooked up to 
> anything.  The Fair Scheduler could make use of a similar option as well.  
> This is a feature-parity regression from MR1.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (YARN-899) Get queue administration ACLs working

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/YARN-899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13734967#comment-13734967
 ] 

Hadoop QA commented on YARN-899:


{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12596979/YARN-899.2.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 6 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:red}-1 javadoc{color}.  The javadoc tool appears to have generated 1 
warning messages.

{color:green}+1 eclipse:eclipse{color}.  The patch built with 
eclipse:eclipse.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 core tests{color}.  The patch failed these unit tests in 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:

  
org.apache.hadoop.yarn.server.resourcemanager.TestClientRMService
  
org.apache.hadoop.yarn.server.resourcemanager.TestApplicationACLs
  
org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesApps
  
org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes
  
org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesFairScheduler
  
org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySched

{color:green}+1 contrib tests{color}.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-YARN-Build/1682//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1682//console

This message is automatically generated.

> Get queue administration ACLs working
> -
>
> Key: YARN-899
> URL: https://issues.apache.org/jira/browse/YARN-899
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: scheduler
>Affects Versions: 2.1.0-beta
>Reporter: Sandy Ryza
>Assignee: Xuan Gong
> Attachments: YARN-899.1.patch, YARN-899.2.patch
>
>
> The Capacity Scheduler documents the 
> yarn.scheduler.capacity.root..acl_administer_queue config option 
> for controlling who can administer a queue, but it is not hooked up to 
> anything.  The Fair Scheduler could make use of a similar option as well.  
> This is a feature-parity regression from MR1.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (YARN-899) Get queue administration ACLs working

[Xuan Gong (JIRA)]

[ 
https://issues.apache.org/jira/browse/YARN-899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13734164#comment-13734164
 ] 

Xuan Gong commented on YARN-899:


Here is my propose: 
We can create QueueACLsManager which encapsulate the ResourceScheduler, so 
whenever we need to check user's permission, we can provide the UGI, 
queueName(This can be found from RMApp) to the Scheduler (No matter it is 
CapacityScheduler, fairScheduler or FIFOScheduler), and let scheduler to help 
us make the decision. 

For each scheduler, we may need to add a new interface : 
Scheduler#hasAccess(UGI, queueName). The queueName is used to find the correct 
Queue. The reason why I send information back to Scheduler and let scheduler 
make the decision is because 
a. I think all the QueueACLsInfos are collected by the Scheduler, and save in 
its queues (I can not findother places which save the QueueACLs), 
b. even if the queue is re-initiated in the future, we do not need to worry 
about it.


Attached is the patch implements this propose. Please give me the suggestions. 

> Get queue administration ACLs working
> -
>
> Key: YARN-899
> URL: https://issues.apache.org/jira/browse/YARN-899
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: scheduler
>Affects Versions: 2.1.0-beta
>Reporter: Sandy Ryza
>Assignee: Xuan Gong
> Attachments: YARN-899.1.patch, YARN-899.2.patch
>
>
> The Capacity Scheduler documents the 
> yarn.scheduler.capacity.root..acl_administer_queue config option 
> for controlling who can administer a queue, but it is not hooked up to 
> anything.  The Fair Scheduler could make use of a similar option as well.  
> This is a feature-parity regression from MR1.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (YARN-899) Get queue administration ACLs working

[Siddharth Seth (JIRA)]

[ 
https://issues.apache.org/jira/browse/YARN-899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13734040#comment-13734040
 ] 

Siddharth Seth commented on YARN-899:
-

bq.  With this in mind, I think who has access should be based on a union of 
ACLs 
Agree. AMs get ACLs from the RM when they register. That could be a combined 
list along with the queue ACLs. It's up to the AMs to enforce these. Maybe the 
RM proxy could do some of this as well. The MR JobHistoryServer gets ACLs from 
the AM - again it's up to this to enforce them. The RM AppHistoryServer will 
need to do the union though.

Don't have experience with JT ACLs, but it does look like that's doing a union 
as well. View vs Modify ACLs for queues makes sense to me.

> Get queue administration ACLs working
> -
>
> Key: YARN-899
> URL: https://issues.apache.org/jira/browse/YARN-899
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: scheduler
>Affects Versions: 2.1.0-beta
>Reporter: Sandy Ryza
>Assignee: Xuan Gong
> Attachments: YARN-899.1.patch
>
>
> The Capacity Scheduler documents the 
> yarn.scheduler.capacity.root..acl_administer_queue config option 
> for controlling who can administer a queue, but it is not hooked up to 
> anything.  The Fair Scheduler could make use of a similar option as well.  
> This is a feature-parity regression from MR1.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (YARN-899) Get queue administration ACLs working

[Sandy Ryza (JIRA)]

[ 
https://issues.apache.org/jira/browse/YARN-899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13732635#comment-13732635
 ] 

Sandy Ryza commented on YARN-899:
-

bq. So, I think that the applicationACLs should be only for the users who has 
access to QueueA, not for the whole users.
Someone more experienced correct me if I'm wrong here, but I believe the goal 
of queue administration ACLs is to allow admins to delegate responsibility.  So 
if I am a cluster admin and I set up a queue for the marketing department and a 
queue for the engineering department, I might want to allow the head of 
marketing to kill applications in the marketing queue without needing to go 
through me.  With this in mind, I think who has access should be based on a 
union of ACLs - I should be able to kill any application in the marketing queue 
either if I am on the application's ACL or if I am on the queue's ACL.

> Get queue administration ACLs working
> -
>
> Key: YARN-899
> URL: https://issues.apache.org/jira/browse/YARN-899
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: scheduler
>Affects Versions: 2.1.0-beta
>Reporter: Sandy Ryza
>Assignee: Xuan Gong
> Attachments: YARN-899.1.patch
>
>
> The Capacity Scheduler documents the 
> yarn.scheduler.capacity.root..acl_administer_queue config option 
> for controlling who can administer a queue, but it is not hooked up to 
> anything.  The Fair Scheduler could make use of a similar option as well.  
> This is a feature-parity regression from MR1.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (YARN-899) Get queue administration ACLs working

[Sandy Ryza (JIRA)]

[ 
https://issues.apache.org/jira/browse/YARN-899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13732615#comment-13732615
 ] 

Sandy Ryza commented on YARN-899:
-

bq. In case of YARN, a queue is a YARN construct. Queue acls should be enforced 
by YARN and not by each scheduler.
Agreed.  We will need to change this for the existing submit ACLs as well.  
Currently the scheduler holds which applications are in which queues and the 
ACLs for each queue.  I don't think we should change that part, so perhaps an 
interface like Scheduler#getQueueACLs(ApplicationAttemptId) would make sense?

bq. There are separate acls for who can view as compared to who can modify ( 
kill, etc ).
As far as I can tell this is not how it worked for queue ACLs in MR1, but I 
think this is how we should do it for MR2.

> Get queue administration ACLs working
> -
>
> Key: YARN-899
> URL: https://issues.apache.org/jira/browse/YARN-899
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: scheduler
>Affects Versions: 2.1.0-beta
>Reporter: Sandy Ryza
>Assignee: Xuan Gong
> Attachments: YARN-899.1.patch
>
>
> The Capacity Scheduler documents the 
> yarn.scheduler.capacity.root..acl_administer_queue config option 
> for controlling who can administer a queue, but it is not hooked up to 
> anything.  The Fair Scheduler could make use of a similar option as well.  
> This is a feature-parity regression from MR1.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (YARN-899) Get queue administration ACLs working

[Hitesh Shah (JIRA)]

[ 
https://issues.apache.org/jira/browse/YARN-899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13732576#comment-13732576
 ] 

Hitesh Shah commented on YARN-899:
--

[~acmurthy] [~vinodkv] can probably correct me as I will likely make some 
mistakes.

>From what I know, acls can control both view and modification abilities. There 
>are separate acls for who can view as compared to who can modify ( kill, etc ).

In case of YARN, a queue is a YARN construct. Queue acls should be enforced by 
YARN and not by each scheduler. I know that this is a problem as queue 
configuration is scheduler specific but we need to look at providing a global 
solution to acls and see how that works with the configured queues in 
themselves. YARN already has cluster level acls for applications. Queue acls 
would be a layer above it but should be available and enforcable regardless of 
what scheduler is used.


 

> Get queue administration ACLs working
> -
>
> Key: YARN-899
> URL: https://issues.apache.org/jira/browse/YARN-899
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: scheduler
>Affects Versions: 2.1.0-beta
>Reporter: Sandy Ryza
>Assignee: Xuan Gong
> Attachments: YARN-899.1.patch
>
>
> The Capacity Scheduler documents the 
> yarn.scheduler.capacity.root..acl_administer_queue config option 
> for controlling who can administer a queue, but it is not hooked up to 
> anything.  The Fair Scheduler could make use of a similar option as well.  
> This is a feature-parity regression from MR1.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (YARN-899) Get queue administration ACLs working

[Xuan Gong (JIRA)]

[ 
https://issues.apache.org/jira/browse/YARN-899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13732287#comment-13732287
 ] 

Xuan Gong commented on YARN-899:


bq. If we can't apply them at this level, then I think the view ACLs are 
something we should leave out.

When the client send an application, the client should provide the queue name 
(say, QueueA) as well as the applicationACLs. So, I think that the 
applicationACLs should be only for the users who has access to QueueA, not for 
the whole users. If someone wants to view the applications on QueueA, no matter 
for details or not, should we need to check whether this user has permission to 
access QueueA first ?? Only using applicationACLsManager to check 
ApplicationAccessType for the user is enough ?

> Get queue administration ACLs working
> -
>
> Key: YARN-899
> URL: https://issues.apache.org/jira/browse/YARN-899
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: scheduler
>Affects Versions: 2.1.0-beta
>Reporter: Sandy Ryza
>Assignee: Xuan Gong
> Attachments: YARN-899.1.patch
>
>
> The Capacity Scheduler documents the 
> yarn.scheduler.capacity.root..acl_administer_queue config option 
> for controlling who can administer a queue, but it is not hooked up to 
> anything.  The Fair Scheduler could make use of a similar option as well.  
> This is a feature-parity regression from MR1.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (YARN-899) Get queue administration ACLs working

[Sandy Ryza (JIRA)]

[ 
https://issues.apache.org/jira/browse/YARN-899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13732176#comment-13732176
 ] 

Sandy Ryza commented on YARN-899:
-

I just took a look at the MR1 code.  There, what was controlled was the ability 
to look at the details of the job.  In MR2, these details are served by the 
JobHistoryServer and app master, so the RM doesn't really have the ability to 
grant or deny access to them.  Or is there a mechanism I'm missing?  If we 
can't apply them at this level, then I think the view ACLs are something we 
should leave out.


> Get queue administration ACLs working
> -
>
> Key: YARN-899
> URL: https://issues.apache.org/jira/browse/YARN-899
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: scheduler
>Affects Versions: 2.1.0-beta
>Reporter: Sandy Ryza
>Assignee: Xuan Gong
> Attachments: YARN-899.1.patch
>
>
> The Capacity Scheduler documents the 
> yarn.scheduler.capacity.root..acl_administer_queue config option 
> for controlling who can administer a queue, but it is not hooked up to 
> anything.  The Fair Scheduler could make use of a similar option as well.  
> This is a feature-parity regression from MR1.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (YARN-899) Get queue administration ACLs working

[Xuan Gong (JIRA)]

[ 
https://issues.apache.org/jira/browse/YARN-899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13732148#comment-13732148
 ] 

Xuan Gong commented on YARN-899:


Thanks for the comments. Sandy.

bq.We should go with an approach that generalizes for all schedulers.

Yes, we should do that. How about using a more generalized type : Queue. Looks 
like that Queue is the super type for all the csqueue, FSQueue, and queue used 
in all three schedulers.

bq.Queue submission ACLs are enforced internally by the schedulers on 
addApplication. Would it make sense to use a similar approach with 
removeApplication? If not, the submission ACLs should probably be modified to 
use whatever mechanism is added.

We can do that

bq.In MR1, did the administer ACLs control who could view jobs in a queue? My 
impression was that they only controlled who could kill jobs and change their 
priorities.

Yes, based on the documentation, it shows "queue administrators of the queue to 
which the job was submitted to (mapred.queue.queue-name.acl-administer-jobs) 
always have access to view and modify a job ". 





> Get queue administration ACLs working
> -
>
> Key: YARN-899
> URL: https://issues.apache.org/jira/browse/YARN-899
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: scheduler
>Affects Versions: 2.1.0-beta
>Reporter: Sandy Ryza
>Assignee: Xuan Gong
> Attachments: YARN-899.1.patch
>
>
> The Capacity Scheduler documents the 
> yarn.scheduler.capacity.root..acl_administer_queue config option 
> for controlling who can administer a queue, but it is not hooked up to 
> anything.  The Fair Scheduler could make use of a similar option as well.  
> This is a feature-parity regression from MR1.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (YARN-899) Get queue administration ACLs working

[Sandy Ryza (JIRA)]

[ 
https://issues.apache.org/jira/browse/YARN-899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13731729#comment-13731729
 ] 

Sandy Ryza commented on YARN-899:
-

Thanks for taking this up, Xuan.  I took a look at the patch.  It appears to 
put Capacity Scheduler-specific code in the RM outside the Capacity Scheduler.  
The added QueueACLsManager, which is referenced in many places in the RM, holds 
a mapping between application IDs and CSQueues.  We should go with an approach 
that generalizes for all schedulers.

Queue submission ACLs are enforced internally by the schedulers on 
addApplication.  Would it make sense to use a similar approach with 
removeApplication?  If not, the submission ACLs should probably be modified to 
use whatever mechanism is added.

In MR1, did the administer ACLs control who could view jobs in a queue?  My 
impression was that they only controlled who could kill jobs and change their 
priorities.

> Get queue administration ACLs working
> -
>
> Key: YARN-899
> URL: https://issues.apache.org/jira/browse/YARN-899
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: scheduler
>Affects Versions: 2.1.0-beta
>Reporter: Sandy Ryza
>Assignee: Xuan Gong
> Attachments: YARN-899.1.patch
>
>
> The Capacity Scheduler documents the 
> yarn.scheduler.capacity.root..acl_administer_queue config option 
> for controlling who can administer a queue, but it is not hooked up to 
> anything.  The Fair Scheduler could make use of a similar option as well.  
> This is a feature-parity regression from MR1.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (YARN-899) Get queue administration ACLs working

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/YARN-899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13731673#comment-13731673
 ] 

Hadoop QA commented on YARN-899:


{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12596497/YARN-899.1.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 13 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 eclipse:eclipse{color}.  The patch built with 
eclipse:eclipse.

{color:red}-1 findbugs{color}.  The patch appears to introduce 1 new 
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.

{color:green}+1 contrib tests{color}.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-YARN-Build/1666//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-YARN-Build/1666//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1666//console

This message is automatically generated.

> Get queue administration ACLs working
> -
>
> Key: YARN-899
> URL: https://issues.apache.org/jira/browse/YARN-899
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: scheduler
>Affects Versions: 2.1.0-beta
>Reporter: Sandy Ryza
>Assignee: Xuan Gong
> Attachments: YARN-899.1.patch
>
>
> The Capacity Scheduler documents the 
> yarn.scheduler.capacity.root..acl_administer_queue config option 
> for controlling who can administer a queue, but it is not hooked up to 
> anything.  The Fair Scheduler could make use of a similar option as well.  
> This is a feature-parity regression from MR1.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (YARN-899) Get queue administration ACLs working

[Xuan Gong (JIRA)]

[ 
https://issues.apache.org/jira/browse/YARN-899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13731602#comment-13731602
 ] 

Xuan Gong commented on YARN-899:


Create a QueueACLsManager to save . Whenever the users 
try to get the applicationReport, list applications, kill applications thru 
commandLine, webservice or UI, queueACLsManager will check users' permission. 


> Get queue administration ACLs working
> -
>
> Key: YARN-899
> URL: https://issues.apache.org/jira/browse/YARN-899
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: scheduler
>Affects Versions: 2.1.0-beta
>Reporter: Sandy Ryza
>Assignee: Xuan Gong
> Attachments: YARN-899.1.patch
>
>
> The Capacity Scheduler documents the 
> yarn.scheduler.capacity.root..acl_administer_queue config option 
> for controlling who can administer a queue, but it is not hooked up to 
> anything.  The Fair Scheduler could make use of a similar option as well.  
> This is a feature-parity regression from MR1.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira