Tao Yang created YARN-11798:
-------------------------------
Summary: Precheck request separately to avoid redundant node
checks and optimize performance for global scheduler.
Key: YARN-11798
URL: https://issues.apache.org/jira/browse/YARN-11798
Project: Hadoop YARN
Issue Type: Improvement
Components: capacityscheduler
Reporter: Tao Yang
Assignee: Tao Yang
The current implementation of the
RegularContainerAllocator#preCheckCandidateNodes performs redundant
check-request logic for each node, leading to significant latency during global
scheduling with a large number of candidate nodes.
By splitting this method into two separate methods: preCheckRequest and
preCheckNode, we can ensure that the check-request logic is executed only once
per request before checking all nodes, thereby optimizing scheduling
performance and reducing latency.
In a scheduling scenario with 2000 candidate nodes and an unsatisfied request
that reaches the headroom, the global scheduler currently checks the request
for each node repeatedly, causing a scheduling cycle to exceed 200ms. With this
improvement, the global scheduler checks the request only once before
evaluating all nodes, reducing the scheduling cycle to less than 2ms.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
