[
https://issues.apache.org/jira/browse/YARN-10514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zhuqi updated YARN-10514:
-
Description:
When we schedule in multi node lookup policy for async scheduling, or just use
heartbeat update based scheduling, we both meet scheduling fragments. When
cpu-intensive jobs or gpu-intensive or memory-intensive etc, the cluster will
meet heavy waste of resources, so this issue will help to move scheduler
support dominant resource based schedule, to help our cluster get better
resource utilization, also in order to load balance nodemanager resource
distribution.
was:When we schedule in multi node lookup policy for async scheduling, or
just use heartbeat update based scheduling, we both meet scheduling fragments.
When cpu-intensive jobs or gpu-intensive or memory-intensive etc, the cluster
will meet heavy waste of resources, so this issue will help to move scheduler
support dominant resource based schedule, to help our cluster get better
resource utilization.
> Introduce a dominant resource based schedule policy to increase the resource
> utilization, avoid heavy cluster resource fragments.
> -
>
> Key: YARN-10514
> URL: https://issues.apache.org/jira/browse/YARN-10514
> Project: Hadoop YARN
> Issue Type: Improvement
>Affects Versions: 3.3.0, 3.4.0
>Reporter: zhuqi
>Assignee: zhuqi
>Priority: Major
> Attachments: YARN-10514.001.patch
>
>
> When we schedule in multi node lookup policy for async scheduling, or just
> use heartbeat update based scheduling, we both meet scheduling fragments.
> When cpu-intensive jobs or gpu-intensive or memory-intensive etc, the cluster
> will meet heavy waste of resources, so this issue will help to move scheduler
> support dominant resource based schedule, to help our cluster get better
> resource utilization, also in order to load balance nodemanager resource
> distribution.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
