[
https://issues.apache.org/jira/browse/YARN-11798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shilun Fan resolved YARN-11798.
-------------------------------
Fix Version/s: 3.5.0
Hadoop Flags: Reviewed
Target Version/s: 3.5.0
Resolution: Fixed
> Precheck request separately to avoid redundant node checks and optimize
> performance for global scheduler.
> ---------------------------------------------------------------------------------------------------------
>
> Key: YARN-11798
> URL: https://issues.apache.org/jira/browse/YARN-11798
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: capacityscheduler
> Reporter: Tao Yang
> Assignee: Tao Yang
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.5.0
>
>
> The current implementation of the
> RegularContainerAllocator#preCheckCandidateNodes performs redundant
> check-request logic for each node, leading to significant latency during
> global scheduling with a large number of candidate nodes.
> By splitting this method into two separate methods: preCheckRequest and
> preCheckNode, we can ensure that the check-request logic is executed only
> once per request before checking all nodes, thereby optimizing scheduling
> performance and reducing latency.
>
> In a scheduling scenario with 2000 candidate nodes and an unsatisfied request
> that reaches the headroom, the global scheduler currently checks the request
> for each node repeatedly, causing a scheduling cycle to exceed 200ms. With
> this improvement, the global scheduler checks the request only once before
> evaluating all nodes, reducing the scheduling cycle to less than 2ms.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
