[jira] [Updated] (YARN-11108) Unexpected preemptions happen when hierarchy queues case

2023-03-22 Thread Juanjuan Tian (Jira) 


 [ 
https://issues.apache.org/jira/browse/YARN-11108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Juanjuan Tian  updated YARN-11108:
--
Description: 
Found unexpected preemptions happen when hierarchy queues case, the issue is 
that a sub queue can accept resource more than used+pending, leading to other 
queues {color:#172b4d}IDEAL_ASSIGNED is smaller than used + pending, and 
preemptions happen unexpectedly {color}
 
2022-04-02T01:11:12,973 DEBUG [SchedulingMonitor 
(ProportionalCapacityPreemptionPolicy)] 
org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.PreemptableResourceCalculator:
  NAME: MSANRPAB PARTITION: persistent{color:#de350b} CUR:  PEN:  TOTAL_PEN:  
RESERVED:  GAR:  NORM: 
0.3424696922302246{color:#de350b} IDEAL_ASSIGNED: {color} IDEAL_PREEMPT:  
ACTUAL_PREEMPT:  UNTOUCHABLE:  PREEMPTABLE: ]> PEN:  TOTAL_PEN:  RESERVED: 
 GAR:  NORM: 0.3424696922302246{color:#de350b} IDEAL_ASSIGNED: 
{color} IDEAL_PREEMPT:  ACTUAL_PREEMPT:  
UNTOUCHABLE:  PREEMPTABLE: ]> BONUS_WEIGHT: 
-1.0
 


> Unexpected preemptions happen when hierarchy queues case
> 
>
> Key: YARN-11108
> URL: https://issues.apache.org/jira/browse/YARN-11108
> Project: Hadoop YARN
>  Issue Type: Improvement
>Affects Versions: 2.9.2
>Reporter: Juanjuan Tian 
>Assignee: Juanjuan Tian 
>Priority: Major
> Attachments: image-2022-04-06-16-29-57-871.png
>
>
> Found unexpected preemptions happen when hierarchy queues case, the issue is 
> that a sub queue can accept resource more than used+pending, leading to other 
> queues {color:#172b4d}IDEAL_ASSIGNED is smaller than used + pending, and 
> preemptions happen unexpectedly {color}
>  
> 2022-04-02T01:11:12,973 DEBUG [SchedulingMonitor 
> (ProportionalCapacityPreemptionPolicy)] 
> org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.PreemptableResourceCalculator:
>   NAME: MSANRPAB PARTITION: persistent{color:#de350b} CUR:  vCores:8285{color}> PEN:  TOTAL_PEN:  
> RESERVED:  GAR:  NORM: 
> 0.3424696922302246{color:#de350b} IDEAL_ASSIGNED:  vCores:8903>{color} IDEAL_PREEMPT:  
> ACTUAL_PREEMPT:  UNTOUCHABLE:  vCores:0, ports:null> PREEMPTABLE:  [  
> from logs, we can see,  {color:#de350b}IDEAL_ASSIGNED is bigger than 
> CUR{color}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Updated] (YARN-11108) Unexpected preemptions happen when hierarchy queues case

2022-04-10 Thread Juanjuan Tian (Jira) 


 [ 
https://issues.apache.org/jira/browse/YARN-11108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Juanjuan Tian  updated YARN-11108:
--
Description: 
Found unexpected preemptions happen when hierarchy queues case, the issue is 
that a sub queue can accept resource more than used+pending, leading to other 
queues {color:#172b4d}IDEAL_ASSIGNED is smaller than used + pending, and 
preemptions happen unexpectedly {color}
 
2022-04-02T01:11:12,973 DEBUG [SchedulingMonitor 
(ProportionalCapacityPreemptionPolicy)] 
org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.PreemptableResourceCalculator:
  NAME: MSANRPAB PARTITION: persistent{color:#de350b} CUR: ]> PEN:  TOTAL_PEN:  RESERVED: 
 GAR:  NORM: 0.3424696922302246{color:#de350b} IDEAL_ASSIGNED: 
{color} IDEAL_PREEMPT:  ACTUAL_PREEMPT:  
UNTOUCHABLE:  PREEMPTABLE: ]> BONUS_WEIGHT: 
-1.0
 

  was:
Found unexpected preemptions happen when hierarchy queues case, the issue is 
that a sub queue can accept resource more than used+pending, leading to other 
queues {color:#172b4d}IDEAL_ASSIGNED is smaller than used + pending, and 
preemption happen unexpectedly {color}
 
2022-04-02T01:11:12,973 DEBUG [SchedulingMonitor 
(ProportionalCapacityPreemptionPolicy)] 
org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.PreemptableResourceCalculator:
  NAME: MSANRPAB PARTITION: persistent{color:#de350b} CUR: ]> PEN:  TOTAL_PEN:  RESERVED: 
 GAR:  NORM: 0.3424696922302246{color:#de350b} IDEAL_ASSIGNED: 
{color} IDEAL_PREEMPT:  ACTUAL_PREEMPT:  
UNTOUCHABLE:  PREEMPTABLE: ]> BONUS_WEIGHT: 
-1.0
 


> Unexpected preemptions happen when hierarchy queues case
> 
>
> Key: YARN-11108
> URL: https://issues.apache.org/jira/browse/YARN-11108
> Project: Hadoop YARN
>  Issue Type: Improvement
>Affects Versions: 2.9.2
>Reporter: Juanjuan Tian 
>Assignee: Juanjuan Tian 
>Priority: Major
> Attachments: image-2022-04-06-16-29-57-871.png
>
>
> Found unexpected preemptions happen when hierarchy queues case, the issue is 
> that a sub queue can accept resource more than used+pending, leading to other 
> queues {color:#172b4d}IDEAL_ASSIGNED is smaller than used + pending, and 
> preemptions happen unexpectedly {color}
>  
> 2022-04-02T01:11:12,973 DEBUG [SchedulingMonitor 
> (ProportionalCapacityPreemptionPolicy)] 
> org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.PreemptableResourceCalculator:
>   NAME: MSANRPAB PARTITION: persistent{color:#de350b} CUR:  vCores:8285, ports:null{color}, [ reservedAffinity:\\{6, 8, 9, 10, 11, 15, 19, 20, 22, 24, 28}>]> PEN: 
>  TOTAL_PEN:  
> RESERVED:  GAR:  vCores:9571, ports:null> NORM: 0.3424696922302246{color:#de350b} 
> IDEAL_ASSIGNED: {color} 
> IDEAL_PREEMPT:  ACTUAL_PREEMPT:  vCores:0, ports:null> UNTOUCHABLE:  
> PREEMPTABLE:  availableCpuCount:-36467, reservedAffinity:\\{6, 8, 9, 10, 11, 15, 19, 20, 
> 22, 24, 28}>]> BONUS_WEIGHT: -1.0
>  



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Updated] (YARN-11108) Unexpected preemptions happen when hierarchy queues case

2022-04-06 Thread Juanjuan Tian (Jira) 


 [ 
https://issues.apache.org/jira/browse/YARN-11108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Juanjuan Tian  updated YARN-11108:
--
Affects Version/s: 2.9.2

> Unexpected preemptions happen when hierarchy queues case
> 
>
> Key: YARN-11108
> URL: https://issues.apache.org/jira/browse/YARN-11108
> Project: Hadoop YARN
>  Issue Type: Improvement
>Affects Versions: 2.9.2
>Reporter: Juanjuan Tian 
>Assignee: Juanjuan Tian 
>Priority: Major
> Attachments: image-2022-04-06-16-29-57-871.png
>
>
> Found unexpected preemptions happen when hierarchy queues case, the issue is 
> that a sub queue can accept resource more than used+pending, leading to other 
> queues {color:#172b4d}IDEAL_ASSIGNED is smaller than used + pending, and 
> preemption happen unexpectedly {color}
>  
> 2022-04-02T01:11:12,973 DEBUG [SchedulingMonitor 
> (ProportionalCapacityPreemptionPolicy)] 
> org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.PreemptableResourceCalculator:
>   NAME: MSANRPAB PARTITION: persistent{color:#de350b} CUR:  vCores:8285, ports:null{color}, [ reservedAffinity:\{6, 8, 9, 10, 11, 15, 19, 20, 22, 24, 28}>]> PEN: 
>  TOTAL_PEN:  
> RESERVED:  GAR:  vCores:9571, ports:null> NORM: 0.3424696922302246{color:#de350b} 
> IDEAL_ASSIGNED: {color} 
> IDEAL_PREEMPT:  ACTUAL_PREEMPT:  vCores:0, ports:null> UNTOUCHABLE:  
> PREEMPTABLE:  availableCpuCount:-36467, reservedAffinity:\{6, 8, 9, 10, 11, 15, 19, 20, 22, 
> 24, 28}>]> BONUS_WEIGHT: -1.0
>  



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Updated] (YARN-11108) Unexpected preemptions happen when hierarchy queues case

2022-04-06 Thread Juanjuan Tian (Jira) 


 [ 
https://issues.apache.org/jira/browse/YARN-11108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Juanjuan Tian  updated YARN-11108:
--
Attachment: image-2022-04-06-16-29-57-871.png

> Unexpected preemptions happen when hierarchy queues case
> 
>
> Key: YARN-11108
> URL: https://issues.apache.org/jira/browse/YARN-11108
> Project: Hadoop YARN
>  Issue Type: Improvement
>Reporter: Juanjuan Tian 
>Priority: Major
> Attachments: image-2022-04-06-16-29-57-871.png
>
>
> Found unexpected preemptions happen when hierarchy queues case, the issue is 
> that a sub queue can accept resource more than used+pending, leading to other 
> queues {color:#172b4d}IDEAL_ASSIGNED is smaller than used + pending, and 
> preemption happen unexpectedly {color}
>  
> 2022-04-02T01:11:12,973 DEBUG [SchedulingMonitor 
> (ProportionalCapacityPreemptionPolicy)] 
> org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.PreemptableResourceCalculator:
>   NAME: MSANRPAB PARTITION: persistent{color:#de350b} CUR:  vCores:8285, ports:null{color}, [ reservedAffinity:\{6, 8, 9, 10, 11, 15, 19, 20, 22, 24, 28}>]> PEN: 
>  TOTAL_PEN:  
> RESERVED:  GAR:  vCores:9571, ports:null> NORM: 0.3424696922302246{color:#de350b} 
> IDEAL_ASSIGNED: {color} 
> IDEAL_PREEMPT:  ACTUAL_PREEMPT:  vCores:0, ports:null> UNTOUCHABLE:  
> PREEMPTABLE:  availableCpuCount:-36467, reservedAffinity:\{6, 8, 9, 10, 11, 15, 19, 20, 22, 
> 24, 28}>]> BONUS_WEIGHT: -1.0
>  



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org