[ https://issues.apache.org/jira/browse/YARN-11108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Juanjuan Tian updated YARN-11108: ---------------------------------- Description: Found unexpected preemptions happen when hierarchy queues case, the issue is that a sub queue can accept resource more than used+pending, leading to other queues {color:#172b4d}IDEAL_ASSIGNED is smaller than used + pending, and preemptions happen unexpectedly {color} 2022-04-02T01:11:12,973 DEBUG [SchedulingMonitor (ProportionalCapacityPreemptionPolicy)] org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.PreemptableResourceCalculator: NAME: MSANRPAB PARTITION: persistent{color:#de350b} CUR: <memory:37934080, vCores:8285{color}> PEN: <memory:0, vCores:0> TOTAL_PEN: <memory:0, vCores:0> RESERVED: <memory:0, vCores:0> GAR: <memory:30720000, vCores:9571> NORM: 0.3424696922302246{color:#de350b} IDEAL_ASSIGNED: <memory:37934080, vCores:8903>{color} IDEAL_PREEMPT: <memory:0, vCores:0, ports:null> ACTUAL_PREEMPT: <memory:0, vCores:0, ports:null> UNTOUCHABLE: <memory:0, vCores:0, ports:null> PREEMPTABLE: <memory:7214080, vCores:-1286, ports:null, [<groupIndex:1, availableCpuCount:-36467, from logs, we can see, {color:#de350b}IDEAL_ASSIGNED is bigger than CUR{color} was: Found unexpected preemptions happen when hierarchy queues case, the issue is that a sub queue can accept resource more than used+pending, leading to other queues {color:#172b4d}IDEAL_ASSIGNED is smaller than used + pending, and preemptions happen unexpectedly {color} 2022-04-02T01:11:12,973 DEBUG [SchedulingMonitor (ProportionalCapacityPreemptionPolicy)] org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.PreemptableResourceCalculator: NAME: MSANRPAB PARTITION: persistent{color:#de350b} CUR: <memory:37934080, vCores:8285, ports:null{color}, [<groupIndex:1, availableCpuCount:-36467, reservedAffinity:\\{6, 8, 9, 10, 11, 15, 19, 20, 22, 24, 28}>]> PEN: <memory:0, vCores:0, ports:null> TOTAL_PEN: <memory:0, vCores:0, ports:null> RESERVED: <memory:0, vCores:0, ports:null> GAR: <memory:30720000, vCores:9571, ports:null> NORM: 0.3424696922302246{color:#de350b} IDEAL_ASSIGNED: <memory:37934080, vCores:8903, ports:null>{color} IDEAL_PREEMPT: <memory:0, vCores:0, ports:null> ACTUAL_PREEMPT: <memory:0, vCores:0, ports:null> UNTOUCHABLE: <memory:0, vCores:0, ports:null> PREEMPTABLE: <memory:7214080, vCores:-1286, ports:null, [<groupIndex:1, availableCpuCount:-36467, reservedAffinity:\\{6, 8, 9, 10, 11, 15, 19, 20, 22, 24, 28}>]> BONUS_WEIGHT: -1.0 > Unexpected preemptions happen when hierarchy queues case > -------------------------------------------------------- > > Key: YARN-11108 > URL: https://issues.apache.org/jira/browse/YARN-11108 > Project: Hadoop YARN > Issue Type: Improvement > Affects Versions: 2.9.2 > Reporter: Juanjuan Tian > Assignee: Juanjuan Tian > Priority: Major > Attachments: image-2022-04-06-16-29-57-871.png > > > Found unexpected preemptions happen when hierarchy queues case, the issue is > that a sub queue can accept resource more than used+pending, leading to other > queues {color:#172b4d}IDEAL_ASSIGNED is smaller than used + pending, and > preemptions happen unexpectedly {color} > > 2022-04-02T01:11:12,973 DEBUG [SchedulingMonitor > (ProportionalCapacityPreemptionPolicy)] > org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.PreemptableResourceCalculator: > NAME: MSANRPAB PARTITION: persistent{color:#de350b} CUR: <memory:37934080, > vCores:8285{color}> PEN: <memory:0, vCores:0> TOTAL_PEN: <memory:0, vCores:0> > RESERVED: <memory:0, vCores:0> GAR: <memory:30720000, vCores:9571> NORM: > 0.3424696922302246{color:#de350b} IDEAL_ASSIGNED: <memory:37934080, > vCores:8903>{color} IDEAL_PREEMPT: <memory:0, vCores:0, ports:null> > ACTUAL_PREEMPT: <memory:0, vCores:0, ports:null> UNTOUCHABLE: <memory:0, > vCores:0, ports:null> PREEMPTABLE: <memory:7214080, vCores:-1286, ports:null, > [<groupIndex:1, availableCpuCount:-36467, > > from logs, we can see, {color:#de350b}IDEAL_ASSIGNED is bigger than > CUR{color} -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org