[jira] [Commented] (YARN-1392) Allow sophisticated app-to-queue placement policies in the Fair Scheduler
[ https://issues.apache.org/jira/browse/YARN-1392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13820848#comment-13820848 ] Sandy Ryza commented on YARN-1392: -- Attached patch. Still need to add doc. > Allow sophisticated app-to-queue placement policies in the Fair Scheduler > - > > Key: YARN-1392 > URL: https://issues.apache.org/jira/browse/YARN-1392 > Project: Hadoop YARN > Issue Type: New Feature > Components: scheduler >Affects Versions: 2.2.0 >Reporter: Sandy Ryza >Assignee: Sandy Ryza > Attachments: YARN-1392.patch > > > Currently the Fair Scheduler supports app-to-queue placement by username. It > would be beneficial to allow more sophisticated policies that rely on primary > and secondary groups and fallbacks. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (YARN-1392) Allow sophisticated app-to-queue placement policies in the Fair Scheduler
[
https://issues.apache.org/jira/browse/YARN-1392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13820889#comment-13820889
]
Hadoop QA commented on YARN-1392:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12613505/YARN-1392.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/2432//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/2432//console
This message is automatically generated.
> Allow sophisticated app-to-queue placement policies in the Fair Scheduler
> -
>
> Key: YARN-1392
> URL: https://issues.apache.org/jira/browse/YARN-1392
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: scheduler
>Affects Versions: 2.2.0
>Reporter: Sandy Ryza
>Assignee: Sandy Ryza
> Attachments: YARN-1392.patch
>
>
> Currently the Fair Scheduler supports app-to-queue placement by username. It
> would be beneficial to allow more sophisticated policies that rely on primary
> and secondary groups and fallbacks.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (YARN-1392) Allow sophisticated app-to-queue placement policies in the Fair Scheduler
[
https://issues.apache.org/jira/browse/YARN-1392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13820979#comment-13820979
]
Hadoop QA commented on YARN-1392:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12613534/YARN-1392-1.patch
against trunk revision .
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/2436//console
This message is automatically generated.
> Allow sophisticated app-to-queue placement policies in the Fair Scheduler
> -
>
> Key: YARN-1392
> URL: https://issues.apache.org/jira/browse/YARN-1392
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: scheduler
>Affects Versions: 2.2.0
>Reporter: Sandy Ryza
>Assignee: Sandy Ryza
> Attachments: YARN-1392-1.patch, YARN-1392.patch
>
>
> Currently the Fair Scheduler supports app-to-queue placement by username. It
> would be beneficial to allow more sophisticated policies that rely on primary
> and secondary groups and fallbacks.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (YARN-1392) Allow sophisticated app-to-queue placement policies in the Fair Scheduler
[ https://issues.apache.org/jira/browse/YARN-1392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13820980#comment-13820980 ] Sandy Ryza commented on YARN-1392: -- Updated patch includes javadoc and fixes "reject" rule handling. > Allow sophisticated app-to-queue placement policies in the Fair Scheduler > - > > Key: YARN-1392 > URL: https://issues.apache.org/jira/browse/YARN-1392 > Project: Hadoop YARN > Issue Type: New Feature > Components: scheduler >Affects Versions: 2.2.0 >Reporter: Sandy Ryza >Assignee: Sandy Ryza > Attachments: YARN-1392-1.patch, YARN-1392.patch > > > Currently the Fair Scheduler supports app-to-queue placement by username. It > would be beneficial to allow more sophisticated policies that rely on primary > and secondary groups and fallbacks. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (YARN-1392) Allow sophisticated app-to-queue placement policies in the Fair Scheduler
[
https://issues.apache.org/jira/browse/YARN-1392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13821134#comment-13821134
]
Hadoop QA commented on YARN-1392:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12613535/YARN-1392-1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/2441//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/2441//console
This message is automatically generated.
> Allow sophisticated app-to-queue placement policies in the Fair Scheduler
> -
>
> Key: YARN-1392
> URL: https://issues.apache.org/jira/browse/YARN-1392
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: scheduler
>Affects Versions: 2.2.0
>Reporter: Sandy Ryza
>Assignee: Sandy Ryza
> Attachments: YARN-1392-1.patch, YARN-1392-1.patch, YARN-1392.patch
>
>
> Currently the Fair Scheduler supports app-to-queue placement by username. It
> would be beneficial to allow more sophisticated policies that rely on primary
> and secondary groups and fallbacks.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (YARN-1392) Allow sophisticated app-to-queue placement policies in the Fair Scheduler
[
https://issues.apache.org/jira/browse/YARN-1392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13821194#comment-13821194
]
Hadoop QA commented on YARN-1392:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12613563/YARN-1392-1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/2442//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/2442//console
This message is automatically generated.
> Allow sophisticated app-to-queue placement policies in the Fair Scheduler
> -
>
> Key: YARN-1392
> URL: https://issues.apache.org/jira/browse/YARN-1392
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: scheduler
>Affects Versions: 2.2.0
>Reporter: Sandy Ryza
>Assignee: Sandy Ryza
> Attachments: YARN-1392-1.patch, YARN-1392-1.patch, YARN-1392.patch
>
>
> Currently the Fair Scheduler supports app-to-queue placement by username. It
> would be beneficial to allow more sophisticated policies that rely on primary
> and secondary groups and fallbacks.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (YARN-1392) Allow sophisticated app-to-queue placement policies in the Fair Scheduler
[
https://issues.apache.org/jira/browse/YARN-1392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13821764#comment-13821764
]
Alejandro Abdelnur commented on YARN-1392:
--
Overall looks good.
* Javadoc warning still surviving:
{code}
[WARNING] Javadoc Warnings
[WARNING]
/home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/QueuePlacementRule.java:75:
warning - @param argument "ugi" is not a parameter name.
{code}
* FairScheduler.apt.vm
** first chunk: s/applicaton/application/
** second chunk: s/subelements/sub-elements/g
** second chunk: not really clear: "which are specified as subelements of the
rule element, for which the argument name is the subelement and the argument
value is its contents."
** second chunk: "The currently existing rules are detailed below:", just say
"Valid rules are:"
** we should spell out the default behavior if create is not specified
* XML syntax for rules, 'type' seems more appropriate than the 'name'
attribute. Said this, wouldn't the following syntax be cleaner:
{code}
{code}
* FairScheduler.java
** Exception message "Application by queue placement policy" not really clear,
how about "Queue placement policy could not resolve to a valid queue"
** {{LOG.error("Error assigning app to queue, placing in default", ex);}}
message is misleading, from what can I tell it will return NULL and it will be
rejected.
* QueuePlacementPolicy.java
** Any special reason for using Maps.newHashMap() instead of JDK HashMap?
** The first constructor is not used, do we need it?
** If the constructor fails to process the policies it throws an exception.
This will abort the complete scheduler config reloading, right? If so, it's OK.
** The parseArgsAndInitialize() method should delegate to each rule class for
parsing its sub-elements (if any), even the create should be handled by the
rule parser.
* {{QueuePlacementRule.java}}
** it seems that if we have a rule with 'create=true' we shouldn't allow follow
up rules as they are never used. The QueryManager should take care of this.
> Allow sophisticated app-to-queue placement policies in the Fair Scheduler
> -
>
> Key: YARN-1392
> URL: https://issues.apache.org/jira/browse/YARN-1392
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: scheduler
>Affects Versions: 2.2.0
>Reporter: Sandy Ryza
>Assignee: Sandy Ryza
> Attachments: YARN-1392-1.patch, YARN-1392-1.patch, YARN-1392.patch
>
>
> Currently the Fair Scheduler supports app-to-queue placement by username. It
> would be beneficial to allow more sophisticated policies that rely on primary
> and secondary groups and fallbacks.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (YARN-1392) Allow sophisticated app-to-queue placement policies in the Fair Scheduler
[ https://issues.apache.org/jira/browse/YARN-1392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13821881#comment-13821881 ] Sandy Ryza commented on YARN-1392: -- Uploading a patch that addresses these comments. > Allow sophisticated app-to-queue placement policies in the Fair Scheduler > - > > Key: YARN-1392 > URL: https://issues.apache.org/jira/browse/YARN-1392 > Project: Hadoop YARN > Issue Type: New Feature > Components: scheduler >Affects Versions: 2.2.0 >Reporter: Sandy Ryza >Assignee: Sandy Ryza > Attachments: YARN-1392-1.patch, YARN-1392-1.patch, YARN-1392-2.patch, > YARN-1392.patch > > > Currently the Fair Scheduler supports app-to-queue placement by username. It > would be beneficial to allow more sophisticated policies that rely on primary > and secondary groups and fallbacks. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (YARN-1392) Allow sophisticated app-to-queue placement policies in the Fair Scheduler
[
https://issues.apache.org/jira/browse/YARN-1392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13821941#comment-13821941
]
Hadoop QA commented on YARN-1392:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12613702/YARN-1392-2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/2448//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/2448//console
This message is automatically generated.
> Allow sophisticated app-to-queue placement policies in the Fair Scheduler
> -
>
> Key: YARN-1392
> URL: https://issues.apache.org/jira/browse/YARN-1392
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: scheduler
>Affects Versions: 2.2.0
>Reporter: Sandy Ryza
>Assignee: Sandy Ryza
> Attachments: YARN-1392-1.patch, YARN-1392-1.patch, YARN-1392-2.patch,
> YARN-1392.patch
>
>
> Currently the Fair Scheduler supports app-to-queue placement by username. It
> would be beneficial to allow more sophisticated policies that rely on primary
> and secondary groups and fallbacks.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (YARN-1392) Allow sophisticated app-to-queue placement policies in the Fair Scheduler
[ https://issues.apache.org/jira/browse/YARN-1392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13821950#comment-13821950 ] Alejandro Abdelnur commented on YARN-1392: -- a few final minor things: * documentation, we should explicitly state that if the specified queue is 'default' it is handled as if no queue has been specified. This behavior is to provide backwards compatibility with how Map-Reduce handles queue setting. * FairScheduler.java, the original logic to map a user to queue if no placement policy is defined seems it could be modeled as a default placement policy rule(s). Then queue resolution is always done by the queue manager. there is a false change (trimming empty line in line 715 after patch) * QueuePlacementPolicy.java, 'primarygroup' rule should be 'primaryGroup' for consistency (update doc as well) > Allow sophisticated app-to-queue placement policies in the Fair Scheduler > - > > Key: YARN-1392 > URL: https://issues.apache.org/jira/browse/YARN-1392 > Project: Hadoop YARN > Issue Type: New Feature > Components: scheduler >Affects Versions: 2.2.0 >Reporter: Sandy Ryza >Assignee: Sandy Ryza > Attachments: YARN-1392-1.patch, YARN-1392-1.patch, YARN-1392-2.patch, > YARN-1392.patch > > > Currently the Fair Scheduler supports app-to-queue placement by username. It > would be beneficial to allow more sophisticated policies that rely on primary > and secondary groups and fallbacks. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (YARN-1392) Allow sophisticated app-to-queue placement policies in the Fair Scheduler
[ https://issues.apache.org/jira/browse/YARN-1392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13822018#comment-13822018 ] Sandy Ryza commented on YARN-1392: -- Uploaded a patch that addresses latest comments. > Allow sophisticated app-to-queue placement policies in the Fair Scheduler > - > > Key: YARN-1392 > URL: https://issues.apache.org/jira/browse/YARN-1392 > Project: Hadoop YARN > Issue Type: New Feature > Components: scheduler >Affects Versions: 2.2.0 >Reporter: Sandy Ryza >Assignee: Sandy Ryza > Attachments: YARN-1392-1.patch, YARN-1392-1.patch, YARN-1392-2.patch, > YARN-1392-3.patch, YARN-1392.patch > > > Currently the Fair Scheduler supports app-to-queue placement by username. It > would be beneficial to allow more sophisticated policies that rely on primary > and secondary groups and fallbacks. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (YARN-1392) Allow sophisticated app-to-queue placement policies in the Fair Scheduler
[
https://issues.apache.org/jira/browse/YARN-1392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13822099#comment-13822099
]
Hadoop QA commented on YARN-1392:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12613743/YARN-1392-3.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/2452//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/2452//console
This message is automatically generated.
> Allow sophisticated app-to-queue placement policies in the Fair Scheduler
> -
>
> Key: YARN-1392
> URL: https://issues.apache.org/jira/browse/YARN-1392
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: scheduler
>Affects Versions: 2.2.0
>Reporter: Sandy Ryza
>Assignee: Sandy Ryza
> Attachments: YARN-1392-1.patch, YARN-1392-1.patch, YARN-1392-2.patch,
> YARN-1392-3.patch, YARN-1392.patch
>
>
> Currently the Fair Scheduler supports app-to-queue placement by username. It
> would be beneficial to allow more sophisticated policies that rely on primary
> and secondary groups and fallbacks.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (YARN-1392) Allow sophisticated app-to-queue placement policies in the Fair Scheduler
[ https://issues.apache.org/jira/browse/YARN-1392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13822939#comment-13822939 ] Alejandro Abdelnur commented on YARN-1392: -- LGTM +1. when committing, please remove 2 false changes in the FairScheduler.java lines 696/1173 (empty lines) > Allow sophisticated app-to-queue placement policies in the Fair Scheduler > - > > Key: YARN-1392 > URL: https://issues.apache.org/jira/browse/YARN-1392 > Project: Hadoop YARN > Issue Type: New Feature > Components: scheduler >Affects Versions: 2.2.0 >Reporter: Sandy Ryza >Assignee: Sandy Ryza > Attachments: YARN-1392-1.patch, YARN-1392-1.patch, YARN-1392-2.patch, > YARN-1392-3.patch, YARN-1392.patch > > > Currently the Fair Scheduler supports app-to-queue placement by username. It > would be beneficial to allow more sophisticated policies that rely on primary > and secondary groups and fallbacks. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (YARN-1392) Allow sophisticated app-to-queue placement policies in the Fair Scheduler
[ https://issues.apache.org/jira/browse/YARN-1392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13823055#comment-13823055 ] Hudson commented on YARN-1392: -- SUCCESS: Integrated in Hadoop-trunk-Commit #4743 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/4743/]) YARN-1392: Add new files (sandy: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1542106) * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/QueuePlacementPolicy.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/QueuePlacementRule.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/SimpleGroupsMapping.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestQueuePlacementPolicy.java YARN-1392. Allow sophisticated app-to-queue placement policies in the Fair Scheduler (Sandy Ryza) (sandy: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1542105) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/QueueManager.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/FairScheduler.apt.vm > Allow sophisticated app-to-queue placement policies in the Fair Scheduler > - > > Key: YARN-1392 > URL: https://issues.apache.org/jira/browse/YARN-1392 > Project: Hadoop YARN > Issue Type: New Feature > Components: scheduler >Affects Versions: 2.2.0 >Reporter: Sandy Ryza >Assignee: Sandy Ryza > Fix For: 2.3.0 > > Attachments: YARN-1392-1.patch, YARN-1392-1.patch, YARN-1392-2.patch, > YARN-1392-3.patch, YARN-1392.patch > > > Currently the Fair Scheduler supports app-to-queue placement by username. It > would be beneficial to allow more sophisticated policies that rely on primary > and secondary groups and fallbacks. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (YARN-1392) Allow sophisticated app-to-queue placement policies in the Fair Scheduler
[ https://issues.apache.org/jira/browse/YARN-1392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13823555#comment-13823555 ] Hudson commented on YARN-1392: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #392 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/392/]) YARN-1392: Add new files (sandy: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1542106) * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/QueuePlacementPolicy.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/QueuePlacementRule.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/SimpleGroupsMapping.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestQueuePlacementPolicy.java YARN-1392. Allow sophisticated app-to-queue placement policies in the Fair Scheduler (Sandy Ryza) (sandy: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1542105) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/QueueManager.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/FairScheduler.apt.vm > Allow sophisticated app-to-queue placement policies in the Fair Scheduler > - > > Key: YARN-1392 > URL: https://issues.apache.org/jira/browse/YARN-1392 > Project: Hadoop YARN > Issue Type: New Feature > Components: scheduler >Affects Versions: 2.2.0 >Reporter: Sandy Ryza >Assignee: Sandy Ryza > Fix For: 2.3.0 > > Attachments: YARN-1392-1.patch, YARN-1392-1.patch, YARN-1392-2.patch, > YARN-1392-3.patch, YARN-1392.patch > > > Currently the Fair Scheduler supports app-to-queue placement by username. It > would be beneficial to allow more sophisticated policies that rely on primary > and secondary groups and fallbacks. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (YARN-1392) Allow sophisticated app-to-queue placement policies in the Fair Scheduler
[ https://issues.apache.org/jira/browse/YARN-1392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13823639#comment-13823639 ] Hudson commented on YARN-1392: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #1583 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1583/]) YARN-1392: Add new files (sandy: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1542106) * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/QueuePlacementPolicy.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/QueuePlacementRule.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/SimpleGroupsMapping.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestQueuePlacementPolicy.java YARN-1392. Allow sophisticated app-to-queue placement policies in the Fair Scheduler (Sandy Ryza) (sandy: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1542105) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/QueueManager.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/FairScheduler.apt.vm > Allow sophisticated app-to-queue placement policies in the Fair Scheduler > - > > Key: YARN-1392 > URL: https://issues.apache.org/jira/browse/YARN-1392 > Project: Hadoop YARN > Issue Type: New Feature > Components: scheduler >Affects Versions: 2.2.0 >Reporter: Sandy Ryza >Assignee: Sandy Ryza > Fix For: 2.3.0 > > Attachments: YARN-1392-1.patch, YARN-1392-1.patch, YARN-1392-2.patch, > YARN-1392-3.patch, YARN-1392.patch > > > Currently the Fair Scheduler supports app-to-queue placement by username. It > would be beneficial to allow more sophisticated policies that rely on primary > and secondary groups and fallbacks. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (YARN-1392) Allow sophisticated app-to-queue placement policies in the Fair Scheduler
[ https://issues.apache.org/jira/browse/YARN-1392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13823658#comment-13823658 ] Hudson commented on YARN-1392: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #1609 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1609/]) YARN-1392: Add new files (sandy: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1542106) * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/QueuePlacementPolicy.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/QueuePlacementRule.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/SimpleGroupsMapping.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestQueuePlacementPolicy.java YARN-1392. Allow sophisticated app-to-queue placement policies in the Fair Scheduler (Sandy Ryza) (sandy: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1542105) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/QueueManager.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/FairScheduler.apt.vm > Allow sophisticated app-to-queue placement policies in the Fair Scheduler > - > > Key: YARN-1392 > URL: https://issues.apache.org/jira/browse/YARN-1392 > Project: Hadoop YARN > Issue Type: New Feature > Components: scheduler >Affects Versions: 2.2.0 >Reporter: Sandy Ryza >Assignee: Sandy Ryza > Fix For: 2.3.0 > > Attachments: YARN-1392-1.patch, YARN-1392-1.patch, YARN-1392-2.patch, > YARN-1392-3.patch, YARN-1392.patch > > > Currently the Fair Scheduler supports app-to-queue placement by username. It > would be beneficial to allow more sophisticated policies that rely on primary > and secondary groups and fallbacks. -- This message was sent by Atlassian JIRA (v6.1#6144)
