Xie YiFan created YARN-11643: -------------------------------- Summary: Skip unnecessary pre-check in Multi Node Placement Key: YARN-11643 URL: https://issues.apache.org/jira/browse/YARN-11643 Project: Hadoop YARN Issue Type: Improvement Components: capacityscheduler Reporter: Xie YiFan Assignee: Xie YiFan
When Multi Node Placement enabled, RegularContainerAllocator do a while loop to find one node from candidate set to allocate for a given scheduler key. Before do allocate, pre-check be called to check if current node satisfies check. If this node does not pass all checks, just continue to next node. {code:java} if (reservedContainer == null) { result = preCheckForNodeCandidateSet(node, schedulingMode, resourceLimits, schedulerKey); if (null != result) { continue; } } {code} But some checks are related to scheduler Key or Application which return PRIORITY_SKIPPED or APP_SKIPPED. It means that if first node does not pass check, the following nodes also do not pass. If cluster have 5000 nodes in default partition, Scheduler will waste 5000 times loop for just one scheduler key. -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org