[
https://issues.apache.org/jira/browse/YARN-11633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17796959#comment-17796959
]
ASF GitHub Bot commented on YARN-11633:
---------------------------------------
slfan1989 commented on code in PR #6356:
URL: https://github.com/apache/hadoop/pull/6356#discussion_r1427433693
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/policies/router/LoadBasedRouterPolicy.java:
##########
@@ -66,12 +66,19 @@ protected SubClusterId chooseSubCluster(
Map<SubClusterIdInfo, Float> weights =
getPolicyInfo().getRouterPolicyWeights();
SubClusterIdInfo chosen = null;
long currBestMem = -1;
+ long currBestVcore = -1;
for (Map.Entry<SubClusterId, SubClusterInfo> entry :
preSelectSubclusters.entrySet()) {
SubClusterIdInfo id = new SubClusterIdInfo(entry.getKey());
if (weights.containsKey(id) && weights.get(id) > 0) {
long availableMemory = getAvailableMemory(entry.getValue());
- if (availableMemory > currBestMem) {
+ long availableVcore = getAvailableVcore(entry.getValue());
+ if (availableMemory > currBestMem && availableVcore > currBestVcore) {
Review Comment:
```
if (availableMemory > currBestMem) {
....
}
```
This code is needed to select the largest subcluster of availableMemory.
Memory is a more strictly limited resource, because without memory, the
application cannot execute. Sufficient memory can also be considered as the
cluster is idle.
If we add cpu constraints (availableVcore), subclusters with smaller memory
may be selected, which is not an expected situation.
> [Federation] Improve LoadBasedRouterPolicy To Use Available vcores
> ------------------------------------------------------------------
>
> Key: YARN-11633
> URL: https://issues.apache.org/jira/browse/YARN-11633
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: federation
> Affects Versions: 3.3.6
> Reporter: yanbin.zhang
> Priority: Major
> Labels: pull-request-available
>
> When selecting a subcluster, consider not only available memory but also
> available vcore
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
