[jira] [Commented] (YARN-2248) Capacity Scheduler changes for moving apps between queues
[ https://issues.apache.org/jira/browse/YARN-2248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14099695#comment-14099695 ] Krisztian Horvath commented on YARN-2248: - We will and let you know if anything seems abnormal. > Capacity Scheduler changes for moving apps between queues > - > > Key: YARN-2248 > URL: https://issues.apache.org/jira/browse/YARN-2248 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler >Reporter: Janos Matyas >Assignee: Janos Matyas > Fix For: 2.6.0 > > Attachments: YARN-2248-1.patch, YARN-2248-2.patch, YARN-2248-3.patch > > > We would like to have the capability (same as the Fair Scheduler has) to move > applications between queues. > We have made a baseline implementation and tests to start with - and we would > like the community to review, come up with suggestions and finally have this > contributed. > The current implementation is available for 2.4.1 - so the first thing is > that we'd need to identify the target version as there are differences > between 2.4.* and 3.* interfaces. > The story behind is available at > http://blog.sequenceiq.com/blog/2014/07/02/move-applications-between-queues/ > and the baseline implementation and test at: > https://github.com/sequenceiq/hadoop-common/blob/branch-2.4.1/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/a/ExtendedCapacityScheduler.java#L924 > https://github.com/sequenceiq/hadoop-common/blob/branch-2.4.1/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/a/TestExtendedCapacitySchedulerAppMove.java -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-2248) Capacity Scheduler changes for moving apps between queues
[ https://issues.apache.org/jira/browse/YARN-2248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14098274#comment-14098274 ] Subramaniam Venkatraman Krishnan commented on YARN-2248: [~jmatyas], [~keyki], we (myself and [~curino]) did some testing on our side. It will be good if you guys also take a look and validate it. Thanks. > Capacity Scheduler changes for moving apps between queues > - > > Key: YARN-2248 > URL: https://issues.apache.org/jira/browse/YARN-2248 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler >Reporter: Janos Matyas >Assignee: Janos Matyas > Fix For: 2.6.0 > > Attachments: YARN-2248-1.patch, YARN-2248-2.patch, YARN-2248-3.patch > > > We would like to have the capability (same as the Fair Scheduler has) to move > applications between queues. > We have made a baseline implementation and tests to start with - and we would > like the community to review, come up with suggestions and finally have this > contributed. > The current implementation is available for 2.4.1 - so the first thing is > that we'd need to identify the target version as there are differences > between 2.4.* and 3.* interfaces. > The story behind is available at > http://blog.sequenceiq.com/blog/2014/07/02/move-applications-between-queues/ > and the baseline implementation and test at: > https://github.com/sequenceiq/hadoop-common/blob/branch-2.4.1/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/a/ExtendedCapacityScheduler.java#L924 > https://github.com/sequenceiq/hadoop-common/blob/branch-2.4.1/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/a/TestExtendedCapacitySchedulerAppMove.java -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-2248) Capacity Scheduler changes for moving apps between queues
[ https://issues.apache.org/jira/browse/YARN-2248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14098235#comment-14098235 ] Jian He commented on YARN-2248: --- I just committed YARN-2378. Thanks [~keyki] and [~jmatyas] for your contributions ! > Capacity Scheduler changes for moving apps between queues > - > > Key: YARN-2248 > URL: https://issues.apache.org/jira/browse/YARN-2248 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler >Reporter: Janos Matyas >Assignee: Janos Matyas > Fix For: 2.6.0 > > Attachments: YARN-2248-1.patch, YARN-2248-2.patch, YARN-2248-3.patch > > > We would like to have the capability (same as the Fair Scheduler has) to move > applications between queues. > We have made a baseline implementation and tests to start with - and we would > like the community to review, come up with suggestions and finally have this > contributed. > The current implementation is available for 2.4.1 - so the first thing is > that we'd need to identify the target version as there are differences > between 2.4.* and 3.* interfaces. > The story behind is available at > http://blog.sequenceiq.com/blog/2014/07/02/move-applications-between-queues/ > and the baseline implementation and test at: > https://github.com/sequenceiq/hadoop-common/blob/branch-2.4.1/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/a/ExtendedCapacityScheduler.java#L924 > https://github.com/sequenceiq/hadoop-common/blob/branch-2.4.1/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/a/TestExtendedCapacitySchedulerAppMove.java -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-2248) Capacity Scheduler changes for moving apps between queues
[ https://issues.apache.org/jira/browse/YARN-2248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14090476#comment-14090476 ] Janos Matyas commented on YARN-2248: Sounds good - let us know if we can help anyhow - we use this feature internally, so once you submit a patch we can check/test on our side as well. > Capacity Scheduler changes for moving apps between queues > - > > Key: YARN-2248 > URL: https://issues.apache.org/jira/browse/YARN-2248 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler >Reporter: Janos Matyas >Assignee: Janos Matyas > Fix For: 2.6.0 > > Attachments: YARN-2248-1.patch, YARN-2248-2.patch, YARN-2248-3.patch > > > We would like to have the capability (same as the Fair Scheduler has) to move > applications between queues. > We have made a baseline implementation and tests to start with - and we would > like the community to review, come up with suggestions and finally have this > contributed. > The current implementation is available for 2.4.1 - so the first thing is > that we'd need to identify the target version as there are differences > between 2.4.* and 3.* interfaces. > The story behind is available at > http://blog.sequenceiq.com/blog/2014/07/02/move-applications-between-queues/ > and the baseline implementation and test at: > https://github.com/sequenceiq/hadoop-common/blob/branch-2.4.1/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/a/ExtendedCapacityScheduler.java#L924 > https://github.com/sequenceiq/hadoop-common/blob/branch-2.4.1/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/a/TestExtendedCapacitySchedulerAppMove.java -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-2248) Capacity Scheduler changes for moving apps between queues
[ https://issues.apache.org/jira/browse/YARN-2248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14089538#comment-14089538 ] Carlo Curino commented on YARN-2248: I work with [~subru] on this... We can run this further on our test clusters (to increase our confidence level before commit), we have a Gridmix harness that among other thing ends up exercising this (once clean we will release as part of YARN-1051). More generally, I agree that merging is the two is the best approach and committing in 2.6.0 sounds good. > Capacity Scheduler changes for moving apps between queues > - > > Key: YARN-2248 > URL: https://issues.apache.org/jira/browse/YARN-2248 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler >Reporter: Janos Matyas >Assignee: Janos Matyas > Fix For: 2.6.0 > > Attachments: YARN-2248-1.patch, YARN-2248-2.patch, YARN-2248-3.patch > > > We would like to have the capability (same as the Fair Scheduler has) to move > applications between queues. > We have made a baseline implementation and tests to start with - and we would > like the community to review, come up with suggestions and finally have this > contributed. > The current implementation is available for 2.4.1 - so the first thing is > that we'd need to identify the target version as there are differences > between 2.4.* and 3.* interfaces. > The story behind is available at > http://blog.sequenceiq.com/blog/2014/07/02/move-applications-between-queues/ > and the baseline implementation and test at: > https://github.com/sequenceiq/hadoop-common/blob/branch-2.4.1/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/a/ExtendedCapacityScheduler.java#L924 > https://github.com/sequenceiq/hadoop-common/blob/branch-2.4.1/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/a/TestExtendedCapacitySchedulerAppMove.java -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-2248) Capacity Scheduler changes for moving apps between queues
[ https://issues.apache.org/jira/browse/YARN-2248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14088912#comment-14088912 ] Wangda Tan commented on YARN-2248: -- [~keyki], I agree we should get move-app committed in 2.6.0. > Capacity Scheduler changes for moving apps between queues > - > > Key: YARN-2248 > URL: https://issues.apache.org/jira/browse/YARN-2248 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler >Reporter: Janos Matyas >Assignee: Janos Matyas > Fix For: 2.6.0 > > Attachments: YARN-2248-1.patch, YARN-2248-2.patch, YARN-2248-3.patch > > > We would like to have the capability (same as the Fair Scheduler has) to move > applications between queues. > We have made a baseline implementation and tests to start with - and we would > like the community to review, come up with suggestions and finally have this > contributed. > The current implementation is available for 2.4.1 - so the first thing is > that we'd need to identify the target version as there are differences > between 2.4.* and 3.* interfaces. > The story behind is available at > http://blog.sequenceiq.com/blog/2014/07/02/move-applications-between-queues/ > and the baseline implementation and test at: > https://github.com/sequenceiq/hadoop-common/blob/branch-2.4.1/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/a/ExtendedCapacityScheduler.java#L924 > https://github.com/sequenceiq/hadoop-common/blob/branch-2.4.1/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/a/TestExtendedCapacitySchedulerAppMove.java -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-2248) Capacity Scheduler changes for moving apps between queues
[ https://issues.apache.org/jira/browse/YARN-2248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14088832#comment-14088832 ] Krisztian Horvath commented on YARN-2248: - Is there a change we can get this committed in 2.6.0? > Capacity Scheduler changes for moving apps between queues > - > > Key: YARN-2248 > URL: https://issues.apache.org/jira/browse/YARN-2248 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler >Reporter: Janos Matyas >Assignee: Janos Matyas > Fix For: 2.6.0 > > Attachments: YARN-2248-1.patch, YARN-2248-2.patch, YARN-2248-3.patch > > > We would like to have the capability (same as the Fair Scheduler has) to move > applications between queues. > We have made a baseline implementation and tests to start with - and we would > like the community to review, come up with suggestions and finally have this > contributed. > The current implementation is available for 2.4.1 - so the first thing is > that we'd need to identify the target version as there are differences > between 2.4.* and 3.* interfaces. > The story behind is available at > http://blog.sequenceiq.com/blog/2014/07/02/move-applications-between-queues/ > and the baseline implementation and test at: > https://github.com/sequenceiq/hadoop-common/blob/branch-2.4.1/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/a/ExtendedCapacityScheduler.java#L924 > https://github.com/sequenceiq/hadoop-common/blob/branch-2.4.1/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/a/TestExtendedCapacitySchedulerAppMove.java -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-2248) Capacity Scheduler changes for moving apps between queues
[ https://issues.apache.org/jira/browse/YARN-2248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14088619#comment-14088619 ] Subramaniam Venkatraman Krishnan commented on YARN-2248: Thanks [~keyki]. I just added all your test cases and ran them & they do pass with my patch including the queue metrics test. The test cases are quite useful, thanks again. > Capacity Scheduler changes for moving apps between queues > - > > Key: YARN-2248 > URL: https://issues.apache.org/jira/browse/YARN-2248 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler >Reporter: Janos Matyas >Assignee: Janos Matyas > Fix For: 2.6.0 > > Attachments: YARN-2248-1.patch, YARN-2248-2.patch, YARN-2248-3.patch > > > We would like to have the capability (same as the Fair Scheduler has) to move > applications between queues. > We have made a baseline implementation and tests to start with - and we would > like the community to review, come up with suggestions and finally have this > contributed. > The current implementation is available for 2.4.1 - so the first thing is > that we'd need to identify the target version as there are differences > between 2.4.* and 3.* interfaces. > The story behind is available at > http://blog.sequenceiq.com/blog/2014/07/02/move-applications-between-queues/ > and the baseline implementation and test at: > https://github.com/sequenceiq/hadoop-common/blob/branch-2.4.1/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/a/ExtendedCapacityScheduler.java#L924 > https://github.com/sequenceiq/hadoop-common/blob/branch-2.4.1/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/a/TestExtendedCapacitySchedulerAppMove.java -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-2248) Capacity Scheduler changes for moving apps between queues
[ https://issues.apache.org/jira/browse/YARN-2248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14088185#comment-14088185 ] Krisztian Horvath commented on YARN-2248: - Hi, As long as we don't break the functionality we can merge them and try to take the best out of them, so yes. Have you tried your patch with the queue metrics test, yet? > Capacity Scheduler changes for moving apps between queues > - > > Key: YARN-2248 > URL: https://issues.apache.org/jira/browse/YARN-2248 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler >Reporter: Janos Matyas >Assignee: Janos Matyas > Fix For: 2.6.0 > > Attachments: YARN-2248-1.patch, YARN-2248-2.patch, YARN-2248-3.patch > > > We would like to have the capability (same as the Fair Scheduler has) to move > applications between queues. > We have made a baseline implementation and tests to start with - and we would > like the community to review, come up with suggestions and finally have this > contributed. > The current implementation is available for 2.4.1 - so the first thing is > that we'd need to identify the target version as there are differences > between 2.4.* and 3.* interfaces. > The story behind is available at > http://blog.sequenceiq.com/blog/2014/07/02/move-applications-between-queues/ > and the baseline implementation and test at: > https://github.com/sequenceiq/hadoop-common/blob/branch-2.4.1/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/a/ExtendedCapacityScheduler.java#L924 > https://github.com/sequenceiq/hadoop-common/blob/branch-2.4.1/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/a/TestExtendedCapacitySchedulerAppMove.java -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-2248) Capacity Scheduler changes for moving apps between queues
[ https://issues.apache.org/jira/browse/YARN-2248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14088137#comment-14088137 ] Subramaniam Venkatraman Krishnan commented on YARN-2248: Hi [~keyki], we have been working on adding support for move for sometime in Capacity Scheduler as part of YARN-2378 (originally YARN-1707) and [~vvasudev] was kind enough to point out that you were doing the same. To prevent duplication, I suggest we merge our work. I looked at your patch & we are doing essentially the same thing (which was good validation for both of us :)). Based on [~leftnoteasy]'s [feedback | https://issues.apache.org/jira/browse/YARN-2378?focusedCommentId=14087723], I think it would be easiest if I merged your metrics test with the patch I have. Would that be OK? > Capacity Scheduler changes for moving apps between queues > - > > Key: YARN-2248 > URL: https://issues.apache.org/jira/browse/YARN-2248 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler >Reporter: Janos Matyas >Assignee: Janos Matyas > Fix For: 2.6.0 > > Attachments: YARN-2248-1.patch, YARN-2248-2.patch, YARN-2248-3.patch > > > We would like to have the capability (same as the Fair Scheduler has) to move > applications between queues. > We have made a baseline implementation and tests to start with - and we would > like the community to review, come up with suggestions and finally have this > contributed. > The current implementation is available for 2.4.1 - so the first thing is > that we'd need to identify the target version as there are differences > between 2.4.* and 3.* interfaces. > The story behind is available at > http://blog.sequenceiq.com/blog/2014/07/02/move-applications-between-queues/ > and the baseline implementation and test at: > https://github.com/sequenceiq/hadoop-common/blob/branch-2.4.1/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/a/ExtendedCapacityScheduler.java#L924 > https://github.com/sequenceiq/hadoop-common/blob/branch-2.4.1/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/a/TestExtendedCapacitySchedulerAppMove.java -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-2248) Capacity Scheduler changes for moving apps between queues
[
https://issues.apache.org/jira/browse/YARN-2248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14054682#comment-14054682
]
Hadoop QA commented on YARN-2248:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12654481/YARN-2248-3.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/4216//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/4216//console
This message is automatically generated.
> Capacity Scheduler changes for moving apps between queues
> -
>
> Key: YARN-2248
> URL: https://issues.apache.org/jira/browse/YARN-2248
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacityscheduler
>Reporter: Janos Matyas
>Assignee: Janos Matyas
> Attachments: YARN-2248-1.patch, YARN-2248-2.patch, YARN-2248-3.patch
>
>
> We would like to have the capability (same as the Fair Scheduler has) to move
> applications between queues.
> We have made a baseline implementation and tests to start with - and we would
> like the community to review, come up with suggestions and finally have this
> contributed.
> The current implementation is available for 2.4.1 - so the first thing is
> that we'd need to identify the target version as there are differences
> between 2.4.* and 3.* interfaces.
> The story behind is available at
> http://blog.sequenceiq.com/blog/2014/07/02/move-applications-between-queues/
> and the baseline implementation and test at:
> https://github.com/sequenceiq/hadoop-common/blob/branch-2.4.1/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/a/ExtendedCapacityScheduler.java#L924
> https://github.com/sequenceiq/hadoop-common/blob/branch-2.4.1/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/a/TestExtendedCapacitySchedulerAppMove.java
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2248) Capacity Scheduler changes for moving apps between queues
[
https://issues.apache.org/jira/browse/YARN-2248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14054566#comment-14054566
]
Hadoop QA commented on YARN-2248:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12654330/YARN-2248-2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/4215//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/4215//console
This message is automatically generated.
> Capacity Scheduler changes for moving apps between queues
> -
>
> Key: YARN-2248
> URL: https://issues.apache.org/jira/browse/YARN-2248
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacityscheduler
>Reporter: Janos Matyas
>Assignee: Janos Matyas
> Attachments: YARN-2248-1.patch, YARN-2248-2.patch
>
>
> We would like to have the capability (same as the Fair Scheduler has) to move
> applications between queues.
> We have made a baseline implementation and tests to start with - and we would
> like the community to review, come up with suggestions and finally have this
> contributed.
> The current implementation is available for 2.4.1 - so the first thing is
> that we'd need to identify the target version as there are differences
> between 2.4.* and 3.* interfaces.
> The story behind is available at
> http://blog.sequenceiq.com/blog/2014/07/02/move-applications-between-queues/
> and the baseline implementation and test at:
> https://github.com/sequenceiq/hadoop-common/blob/branch-2.4.1/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/a/ExtendedCapacityScheduler.java#L924
> https://github.com/sequenceiq/hadoop-common/blob/branch-2.4.1/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/a/TestExtendedCapacitySchedulerAppMove.java
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2248) Capacity Scheduler changes for moving apps between queues
[ https://issues.apache.org/jira/browse/YARN-2248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14053750#comment-14053750 ] Krisztian Horvath commented on YARN-2248: - Can anyone take a look at the patch? I've some concerns regarding the live containers. Movement steps: 1, Check if the target queue has enough capacity and some more validation, exception otherwise (same as with FairScheduler) 2, Remove the app attempt from the current queue 3, Release resources used by live containers on this queue 4, Remove application upwards root (--numApplications) 5, QueueMetrics update 6, Set new queue in application 7, Allocate resources consumed by the live containers (basically the resource usage moved here from the original queue) 8, Submit new app attempt 9, Add application (++numApplications) > Capacity Scheduler changes for moving apps between queues > - > > Key: YARN-2248 > URL: https://issues.apache.org/jira/browse/YARN-2248 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler >Reporter: Janos Matyas >Assignee: Janos Matyas > Attachments: YARN-2248-1.patch, YARN-2248-2.patch > > > We would like to have the capability (same as the Fair Scheduler has) to move > applications between queues. > We have made a baseline implementation and tests to start with - and we would > like the community to review, come up with suggestions and finally have this > contributed. > The current implementation is available for 2.4.1 - so the first thing is > that we'd need to identify the target version as there are differences > between 2.4.* and 3.* interfaces. > The story behind is available at > http://blog.sequenceiq.com/blog/2014/07/02/move-applications-between-queues/ > and the baseline implementation and test at: > https://github.com/sequenceiq/hadoop-common/blob/branch-2.4.1/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/a/ExtendedCapacityScheduler.java#L924 > https://github.com/sequenceiq/hadoop-common/blob/branch-2.4.1/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/a/TestExtendedCapacitySchedulerAppMove.java -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-2248) Capacity Scheduler changes for moving apps between queues
[
https://issues.apache.org/jira/browse/YARN-2248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14052327#comment-14052327
]
Hadoop QA commented on YARN-2248:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12654072/YARN-2248-1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/4203//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/4203//console
This message is automatically generated.
> Capacity Scheduler changes for moving apps between queues
> -
>
> Key: YARN-2248
> URL: https://issues.apache.org/jira/browse/YARN-2248
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacityscheduler
>Reporter: Janos Matyas
>Assignee: Janos Matyas
>Priority: Minor
> Attachments: YARN-2248-1.patch
>
>
> We would like to have the capability (same as the Fair Scheduler has) to move
> applications between queues.
> We have made a baseline implementation and tests to start with - and we would
> like the community to review, come up with suggestions and finally have this
> contributed.
> The current implementation is available for 2.4.1 - so the first thing is
> that we'd need to identify the target version as there are differences
> between 2.4.* and 3.* interfaces.
> The story behind is available at
> http://blog.sequenceiq.com/blog/2014/07/02/move-applications-between-queues/
> and the baseline implementation and test at:
> https://github.com/sequenceiq/hadoop-common/blob/branch-2.4.1/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/a/ExtendedCapacityScheduler.java#L924
> https://github.com/sequenceiq/hadoop-common/blob/branch-2.4.1/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/a/TestExtendedCapacitySchedulerAppMove.java
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2248) Capacity Scheduler changes for moving apps between queues
[ https://issues.apache.org/jira/browse/YARN-2248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14050729#comment-14050729 ] Vinod Kumar Vavilapalli commented on YARN-2248: --- Do you mind attaching a patch against latest YARN trunk? Thanks.. > Capacity Scheduler changes for moving apps between queues > - > > Key: YARN-2248 > URL: https://issues.apache.org/jira/browse/YARN-2248 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler >Reporter: Janos Matyas >Assignee: Janos Matyas >Priority: Minor > > We would like to have the capability (same as the Fair Scheduler has) to move > applications between queues. > We have made a baseline implementation and tests to start with - and we would > like the community to review, come up with suggestions and finally have this > contributed. > The current implementation is available for 2.4.1 - so the first thing is > that we'd need to identify the target version as there are differences > between 2.4.* and 3.* interfaces. > The story behind is available at > http://blog.sequenceiq.com/blog/2014/07/02/move-applications-between-queues/ > and the baseline implementation and test at: > https://github.com/sequenceiq/hadoop-common/blob/branch-2.4.1/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/a/ExtendedCapacityScheduler.java#L924 > https://github.com/sequenceiq/hadoop-common/blob/branch-2.4.1/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/a/TestExtendedCapacitySchedulerAppMove.java -- This message was sent by Atlassian JIRA (v6.2#6252)
