[jira] [Commented] (YARN-4026) FiCaSchedulerApp: ContainerAllocator should be able to choose how to order pending resource requests
[
https://issues.apache.org/jira/browse/YARN-4026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14661459#comment-14661459
]
Hadoop QA commented on YARN-4026:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 15m 11s | Findbugs (version ) appears to
be broken on trunk. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 41s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 42s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 24s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 27s | There were no new checkstyle
issues. |
| {color:red}-1{color} | whitespace | 0m 6s | The patch has 27 line(s) that
end in whitespace. Use git apply --whitespace=fix. |
| {color:green}+1{color} | install | 1m 22s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with
eclipse:eclipse. |
| {color:red}-1{color} | findbugs | 1m 31s | The patch appears to introduce 1
new Findbugs (version 3.0.0) warnings. |
| {color:red}-1{color} | yarn tests | 52m 40s | Tests failed in
hadoop-yarn-server-resourcemanager. |
| | | 89m 39s | |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-yarn-server-resourcemanager |
| Failed unit tests |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation |
| | hadoop.yarn.server.resourcemanager.TestRMAdminService |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12749203/YARN-4026.1.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / b6265d3 |
| whitespace |
https://builds.apache.org/job/PreCommit-YARN-Build/8786/artifact/patchprocess/whitespace.txt
|
| Findbugs warnings |
https://builds.apache.org/job/PreCommit-YARN-Build/8786/artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
|
| hadoop-yarn-server-resourcemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/8786/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/8786/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/8786/console |
This message was automatically generated.
> FiCaSchedulerApp: ContainerAllocator should be able to choose how to order
> pending resource requests
>
>
> Key: YARN-4026
> URL: https://issues.apache.org/jira/browse/YARN-4026
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Wangda Tan
>Assignee: Wangda Tan
> Attachments: YARN-4026.1.patch
>
>
> After YARN-3983, we have an extensible ContainerAllocator which can be used
> by FiCaSchedulerApp to decide how to allocate resources.
> While working on YARN-1651 (allocate resource to increase container), I found
> one thing in existing logic not flexible enough:
> - ContainerAllocator decides what to allocate for a given node and priority:
> To support different kinds of resource allocation, for example, priority as
> weight / skip priority or not, etc. It's better to let ContainerAllocator to
> choose how to order pending resource requests.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4026) FiCaSchedulerApp: ContainerAllocator should be able to choose how to order pending resource requests
[
https://issues.apache.org/jira/browse/YARN-4026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14687398#comment-14687398
]
Jian He commented on YARN-4026:
---
- why "assignment.setFulfilledReservation(true);" is called in Reserved state ?
{code}
if (result.getAllocationState() == AllocationState.RESERVED) {
// This is a reserved container
LOG.info("Reserved container " + " application="
+ application.getApplicationId() + " resource=" + allocatedResource
+ " queue=" + this.toString() + " cluster=" + clusterResource);
assignment.getAssignmentInformation().addReservationDetails(
updatedContainer.getId(),
application.getCSLeafQueue().getQueuePath());
assignment.getAssignmentInformation().incrReservations();
Resources.addTo(assignment.getAssignmentInformation().getReserved(),
allocatedResource);
assignment.setFulfilledReservation(true);
} else {
{code}
- I think here can always return ContainerAllocation.LOCALITY_SKIPPED as the
semantics of this method is to try to allocate a container for certain
locality.
{code}
return type == NodeType.OFF_SWITCH ? ContainerAllocation.APP_SKIPPED
: ContainerAllocation.LOCALITY_SKIPPED;
{code}
The caller here can choose to return APP_SKIPPED if it sees the LOCALITY_SKIPPED
{code}
assigned =
assignOffSwitchContainers(clusterResource, offSwitchResourceRequest,
node, priority, reservedContainer, schedulingMode,
currentResoureLimits);
assigned.requestNodeType = requestType;
return assigned;
}
{code}
> FiCaSchedulerApp: ContainerAllocator should be able to choose how to order
> pending resource requests
>
>
> Key: YARN-4026
> URL: https://issues.apache.org/jira/browse/YARN-4026
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Wangda Tan
>Assignee: Wangda Tan
> Attachments: YARN-4026.1.patch
>
>
> After YARN-3983, we have an extensible ContainerAllocator which can be used
> by FiCaSchedulerApp to decide how to allocate resources.
> While working on YARN-1651 (allocate resource to increase container), I found
> one thing in existing logic not flexible enough:
> - ContainerAllocator decides what to allocate for a given node and priority:
> To support different kinds of resource allocation, for example, priority as
> weight / skip priority or not, etc. It's better to let ContainerAllocator to
> choose how to order pending resource requests.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4026) FiCaSchedulerApp: ContainerAllocator should be able to choose how to order pending resource requests
[
https://issues.apache.org/jira/browse/YARN-4026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14692827#comment-14692827
]
Hadoop QA commented on YARN-4026:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 17m 16s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 8m 18s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 10m 17s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 24s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 0m 54s | The applied patch generated 3
new checkstyle issues (total was 128, now 128). |
| {color:red}-1{color} | whitespace | 0m 5s | The patch has 30 line(s) that
end in whitespace. Use git apply --whitespace=fix. |
| {color:green}+1{color} | install | 1m 27s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 36s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 1m 34s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:red}-1{color} | yarn tests | 53m 47s | Tests failed in
hadoop-yarn-server-resourcemanager. |
| | | 94m 41s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | hadoop.yarn.server.resourcemanager.TestRMAdminService |
| |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12750004/YARN-4026.3.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 7c796fd |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/8827/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt
|
| whitespace |
https://builds.apache.org/job/PreCommit-YARN-Build/8827/artifact/patchprocess/whitespace.txt
|
| hadoop-yarn-server-resourcemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/8827/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/8827/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/8827/console |
This message was automatically generated.
> FiCaSchedulerApp: ContainerAllocator should be able to choose how to order
> pending resource requests
>
>
> Key: YARN-4026
> URL: https://issues.apache.org/jira/browse/YARN-4026
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Wangda Tan
>Assignee: Wangda Tan
> Attachments: YARN-4026.1.patch, YARN-4026.2.patch, YARN-4026.3.patch
>
>
> After YARN-3983, we have an extensible ContainerAllocator which can be used
> by FiCaSchedulerApp to decide how to allocate resources.
> While working on YARN-1651 (allocate resource to increase container), I found
> one thing in existing logic not flexible enough:
> - ContainerAllocator decides what to allocate for a given node and priority:
> To support different kinds of resource allocation, for example, priority as
> weight / skip priority or not, etc. It's better to let ContainerAllocator to
> choose how to order pending resource requests.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4026) FiCaSchedulerApp: ContainerAllocator should be able to choose how to order pending resource requests
[
https://issues.apache.org/jira/browse/YARN-4026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14693023#comment-14693023
]
Hadoop QA commented on YARN-4026:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 16m 21s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 44s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 41s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 0m 49s | The applied patch generated 3
new checkstyle issues (total was 128, now 128). |
| {color:red}-1{color} | whitespace | 0m 5s | The patch has 30 line(s) that
end in whitespace. Use git apply --whitespace=fix. |
| {color:green}+1{color} | install | 1m 20s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 1m 29s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:red}-1{color} | yarn tests | 53m 24s | Tests failed in
hadoop-yarn-server-resourcemanager. |
| | | 91m 53s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12750004/YARN-4026.3.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 3ae716f |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/8829/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt
|
| whitespace |
https://builds.apache.org/job/PreCommit-YARN-Build/8829/artifact/patchprocess/whitespace.txt
|
| hadoop-yarn-server-resourcemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/8829/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/8829/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/8829/console |
This message was automatically generated.
> FiCaSchedulerApp: ContainerAllocator should be able to choose how to order
> pending resource requests
>
>
> Key: YARN-4026
> URL: https://issues.apache.org/jira/browse/YARN-4026
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Wangda Tan
>Assignee: Wangda Tan
> Attachments: YARN-4026.1.patch, YARN-4026.2.patch, YARN-4026.3.patch
>
>
> After YARN-3983, we have an extensible ContainerAllocator which can be used
> by FiCaSchedulerApp to decide how to allocate resources.
> While working on YARN-1651 (allocate resource to increase container), I found
> one thing in existing logic not flexible enough:
> - ContainerAllocator decides what to allocate for a given node and priority:
> To support different kinds of resource allocation, for example, priority as
> weight / skip priority or not, etc. It's better to let ContainerAllocator to
> choose how to order pending resource requests.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4026) FiCaSchedulerApp: ContainerAllocator should be able to choose how to order pending resource requests
[ https://issues.apache.org/jira/browse/YARN-4026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14694341#comment-14694341 ] Hudson commented on YARN-4026: -- FAILURE: Integrated in Hadoop-trunk-Commit #8289 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/8289/]) YARN-4026. Refactored ContainerAllocator to accept a list of priorites rather than a single priority. Contributed by Wangda Tan (jianhe: rev e5003be907acef87c2770e3f2914953f62017b0e) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestLeafQueue.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/allocator/ContainerAllocator.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/common/fica/FiCaSchedulerApp.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/allocator/RegularContainerAllocator.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/allocator/ContainerAllocation.java * hadoop-yarn-project/CHANGES.txt > FiCaSchedulerApp: ContainerAllocator should be able to choose how to order > pending resource requests > > > Key: YARN-4026 > URL: https://issues.apache.org/jira/browse/YARN-4026 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Wangda Tan >Assignee: Wangda Tan > Fix For: 2.8.0 > > Attachments: YARN-4026.1.patch, YARN-4026.2.patch, YARN-4026.3.patch > > > After YARN-3983, we have an extensible ContainerAllocator which can be used > by FiCaSchedulerApp to decide how to allocate resources. > While working on YARN-1651 (allocate resource to increase container), I found > one thing in existing logic not flexible enough: > - ContainerAllocator decides what to allocate for a given node and priority: > To support different kinds of resource allocation, for example, priority as > weight / skip priority or not, etc. It's better to let ContainerAllocator to > choose how to order pending resource requests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4026) FiCaSchedulerApp: ContainerAllocator should be able to choose how to order pending resource requests
[ https://issues.apache.org/jira/browse/YARN-4026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14695120#comment-14695120 ] Hudson commented on YARN-4026: -- FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #286 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/286/]) YARN-4026. Refactored ContainerAllocator to accept a list of priorites rather than a single priority. Contributed by Wangda Tan (jianhe: rev e5003be907acef87c2770e3f2914953f62017b0e) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestLeafQueue.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/common/fica/FiCaSchedulerApp.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/allocator/RegularContainerAllocator.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/allocator/ContainerAllocator.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/allocator/ContainerAllocation.java * hadoop-yarn-project/CHANGES.txt > FiCaSchedulerApp: ContainerAllocator should be able to choose how to order > pending resource requests > > > Key: YARN-4026 > URL: https://issues.apache.org/jira/browse/YARN-4026 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Wangda Tan >Assignee: Wangda Tan > Fix For: 2.8.0 > > Attachments: YARN-4026.1.patch, YARN-4026.2.patch, YARN-4026.3.patch > > > After YARN-3983, we have an extensible ContainerAllocator which can be used > by FiCaSchedulerApp to decide how to allocate resources. > While working on YARN-1651 (allocate resource to increase container), I found > one thing in existing logic not flexible enough: > - ContainerAllocator decides what to allocate for a given node and priority: > To support different kinds of resource allocation, for example, priority as > weight / skip priority or not, etc. It's better to let ContainerAllocator to > choose how to order pending resource requests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4026) FiCaSchedulerApp: ContainerAllocator should be able to choose how to order pending resource requests
[ https://issues.apache.org/jira/browse/YARN-4026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14695127#comment-14695127 ] Hudson commented on YARN-4026: -- FAILURE: Integrated in Hadoop-Yarn-trunk #1016 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/1016/]) YARN-4026. Refactored ContainerAllocator to accept a list of priorites rather than a single priority. Contributed by Wangda Tan (jianhe: rev e5003be907acef87c2770e3f2914953f62017b0e) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/allocator/ContainerAllocation.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestLeafQueue.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/allocator/RegularContainerAllocator.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/allocator/ContainerAllocator.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/common/fica/FiCaSchedulerApp.java > FiCaSchedulerApp: ContainerAllocator should be able to choose how to order > pending resource requests > > > Key: YARN-4026 > URL: https://issues.apache.org/jira/browse/YARN-4026 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Wangda Tan >Assignee: Wangda Tan > Fix For: 2.8.0 > > Attachments: YARN-4026.1.patch, YARN-4026.2.patch, YARN-4026.3.patch > > > After YARN-3983, we have an extensible ContainerAllocator which can be used > by FiCaSchedulerApp to decide how to allocate resources. > While working on YARN-1651 (allocate resource to increase container), I found > one thing in existing logic not flexible enough: > - ContainerAllocator decides what to allocate for a given node and priority: > To support different kinds of resource allocation, for example, priority as > weight / skip priority or not, etc. It's better to let ContainerAllocator to > choose how to order pending resource requests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4026) FiCaSchedulerApp: ContainerAllocator should be able to choose how to order pending resource requests
[ https://issues.apache.org/jira/browse/YARN-4026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14695391#comment-14695391 ] Hudson commented on YARN-4026: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #2213 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2213/]) YARN-4026. Refactored ContainerAllocator to accept a list of priorites rather than a single priority. Contributed by Wangda Tan (jianhe: rev e5003be907acef87c2770e3f2914953f62017b0e) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestLeafQueue.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/allocator/ContainerAllocator.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/allocator/RegularContainerAllocator.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/allocator/ContainerAllocation.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/common/fica/FiCaSchedulerApp.java > FiCaSchedulerApp: ContainerAllocator should be able to choose how to order > pending resource requests > > > Key: YARN-4026 > URL: https://issues.apache.org/jira/browse/YARN-4026 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Wangda Tan >Assignee: Wangda Tan > Fix For: 2.8.0 > > Attachments: YARN-4026.1.patch, YARN-4026.2.patch, YARN-4026.3.patch > > > After YARN-3983, we have an extensible ContainerAllocator which can be used > by FiCaSchedulerApp to decide how to allocate resources. > While working on YARN-1651 (allocate resource to increase container), I found > one thing in existing logic not flexible enough: > - ContainerAllocator decides what to allocate for a given node and priority: > To support different kinds of resource allocation, for example, priority as > weight / skip priority or not, etc. It's better to let ContainerAllocator to > choose how to order pending resource requests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4026) FiCaSchedulerApp: ContainerAllocator should be able to choose how to order pending resource requests
[ https://issues.apache.org/jira/browse/YARN-4026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14695458#comment-14695458 ] Hudson commented on YARN-4026: -- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #275 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/275/]) YARN-4026. Refactored ContainerAllocator to accept a list of priorites rather than a single priority. Contributed by Wangda Tan (jianhe: rev e5003be907acef87c2770e3f2914953f62017b0e) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/allocator/RegularContainerAllocator.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/allocator/ContainerAllocator.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/allocator/ContainerAllocation.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/common/fica/FiCaSchedulerApp.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestLeafQueue.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java * hadoop-yarn-project/CHANGES.txt > FiCaSchedulerApp: ContainerAllocator should be able to choose how to order > pending resource requests > > > Key: YARN-4026 > URL: https://issues.apache.org/jira/browse/YARN-4026 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Wangda Tan >Assignee: Wangda Tan > Fix For: 2.8.0 > > Attachments: YARN-4026.1.patch, YARN-4026.2.patch, YARN-4026.3.patch > > > After YARN-3983, we have an extensible ContainerAllocator which can be used > by FiCaSchedulerApp to decide how to allocate resources. > While working on YARN-1651 (allocate resource to increase container), I found > one thing in existing logic not flexible enough: > - ContainerAllocator decides what to allocate for a given node and priority: > To support different kinds of resource allocation, for example, priority as > weight / skip priority or not, etc. It's better to let ContainerAllocator to > choose how to order pending resource requests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4026) FiCaSchedulerApp: ContainerAllocator should be able to choose how to order pending resource requests
[ https://issues.apache.org/jira/browse/YARN-4026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14695483#comment-14695483 ] Hudson commented on YARN-4026: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #283 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/283/]) YARN-4026. Refactored ContainerAllocator to accept a list of priorites rather than a single priority. Contributed by Wangda Tan (jianhe: rev e5003be907acef87c2770e3f2914953f62017b0e) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/allocator/RegularContainerAllocator.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestLeafQueue.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/common/fica/FiCaSchedulerApp.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/allocator/ContainerAllocation.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/allocator/ContainerAllocator.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java > FiCaSchedulerApp: ContainerAllocator should be able to choose how to order > pending resource requests > > > Key: YARN-4026 > URL: https://issues.apache.org/jira/browse/YARN-4026 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Wangda Tan >Assignee: Wangda Tan > Fix For: 2.8.0 > > Attachments: YARN-4026.1.patch, YARN-4026.2.patch, YARN-4026.3.patch > > > After YARN-3983, we have an extensible ContainerAllocator which can be used > by FiCaSchedulerApp to decide how to allocate resources. > While working on YARN-1651 (allocate resource to increase container), I found > one thing in existing logic not flexible enough: > - ContainerAllocator decides what to allocate for a given node and priority: > To support different kinds of resource allocation, for example, priority as > weight / skip priority or not, etc. It's better to let ContainerAllocator to > choose how to order pending resource requests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4026) FiCaSchedulerApp: ContainerAllocator should be able to choose how to order pending resource requests
[ https://issues.apache.org/jira/browse/YARN-4026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14695522#comment-14695522 ] Hudson commented on YARN-4026: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #2232 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2232/]) YARN-4026. Refactored ContainerAllocator to accept a list of priorites rather than a single priority. Contributed by Wangda Tan (jianhe: rev e5003be907acef87c2770e3f2914953f62017b0e) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestLeafQueue.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/allocator/ContainerAllocator.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/common/fica/FiCaSchedulerApp.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/allocator/RegularContainerAllocator.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/allocator/ContainerAllocation.java > FiCaSchedulerApp: ContainerAllocator should be able to choose how to order > pending resource requests > > > Key: YARN-4026 > URL: https://issues.apache.org/jira/browse/YARN-4026 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Wangda Tan >Assignee: Wangda Tan > Fix For: 2.8.0 > > Attachments: YARN-4026.1.patch, YARN-4026.2.patch, YARN-4026.3.patch > > > After YARN-3983, we have an extensible ContainerAllocator which can be used > by FiCaSchedulerApp to decide how to allocate resources. > While working on YARN-1651 (allocate resource to increase container), I found > one thing in existing logic not flexible enough: > - ContainerAllocator decides what to allocate for a given node and priority: > To support different kinds of resource allocation, for example, priority as > weight / skip priority or not, etc. It's better to let ContainerAllocator to > choose how to order pending resource requests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
