[ https://issues.apache.org/jira/browse/YARN-6769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
daemon updated YARN-6769: ------------------------- Comment: was deleted (was: diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/policies/FairSharePolicy.java b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/policies/FairSharePolicy.java index f8cdb45929..e930b80e45 100644 --- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/policies/FairSharePolicy.java +++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/policies/FairSharePolicy.java @@ -79,6 +79,19 @@ public String getName() { @Override public int compare(Schedulable s1, Schedulable s2) { + Resource demand1 = s1.getDemand(); + Resource demand2 = s2.getDemand(); + // Put the schedulable which does not require resource to + // the end. So the other schedulable can get resource as soon as + // possible though it use resource greater then it minShare or demand. + if (demand1.equals(Resources.none()) && + !demand2.equals(Resources.none())) { + return 1; + } else if (demand2.equals(Resources.none()) && + !demand1.equals(Resources.none())) { + return -1; + } + double minShareRatio1, minShareRatio2; double useToWeightRatio1, useToWeightRatio2; double weight1, weight2; @@ -86,9 +99,9 @@ public int compare(Schedulable s1, Schedulable s2) { Resource resourceUsage1 = s1.getResourceUsage(); Resource resourceUsage2 = s2.getResourceUsage(); Resource minShare1 = Resources.min(RESOURCE_CALCULATOR, null, - s1.getMinShare(), s1.getDemand()); + s1.getMinShare(), demand1); Resource minShare2 = Resources.min(RESOURCE_CALCULATOR, null, - s2.getMinShare(), s2.getDemand()); + s2.getMinShare(), demand2); boolean s1Needy = Resources.lessThan(RESOURCE_CALCULATOR, null, resourceUsage1, minShare1); boolean s2Needy = Resources.lessThan(RESOURCE_CALCULATOR, null, ) > Put the no demand queue after the most in FairSharePolicy#compare > ----------------------------------------------------------------- > > Key: YARN-6769 > URL: https://issues.apache.org/jira/browse/YARN-6769 > Project: Hadoop YARN > Issue Type: Bug > Components: fairscheduler > Affects Versions: 2.7.2 > Reporter: daemon > Priority: Minor > Fix For: 2.9.0 > > > When use fairsheduler as RM scheduler, before assign container we will sort > all queues or applications. > We will use FairSharePolicy#compare as the comparator, but the comparator is > not so perfect. > It have a problem as blow: > 1. when a queue use resource over minShare(minResources), it will put behind > the queue whose demand is zeor. > so it will greater opportunity to get the resource although it do not want. > It will waste schedule time when assign container > to queue or application. > I have fix it, and I will upload the patch to the jira. -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org