Craig Condit created YUNIKORN-847:
-------------------------------------
Summary: Use resource weighting for node sorting
Key: YUNIKORN-847
URL: https://issues.apache.org/jira/browse/YUNIKORN-847
Project: Apache YuniKorn
Issue Type: Improvement
Components: core - scheduler
Reporter: Craig Condit
Assignee: Craig Condit
After YUNIKORN-807, node sorting uses a scoring algorithm which uses the
largest resource share to determine priority. However, resource units are not
taken into account, and consequently, either fair or binpacking tends to look
almost exclusively at the largest absolute values for units. In some
environments this can lead to priority inversions.
We should instead calculate in-use percentages for each resource, and then
weight them to arrive at an overall usage percentage, which can then be used
for scoring. Weights can be controlled at the partition level.
Configuration is as follows:
{code:java}
partitions:
- name: default
nodesortpolicy:
type: fair
resourceweights:
vcore: 4.0
memory: 1.0
queues:
- name: root
submitacl: '*'
{code}
In this example, *vcore* is weighted double what memory is. If
*resourceweights* is not specified, the configuration is equivalent to *{
vcore: 1, memory: 1}.*
For example, in the default configuration, if a node has 60% of its vcores in
use, and 20% of its memory in use, node usage would be calculated at 40% using
equal weights. In the example given above, usage would be calculated as 52%.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
