brumi1024 commented on code in PR #8800:
URL: https://github.com/apache/hadoop/pull/8800#discussion_r4240848613
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java:
##########
@@ -719,9 +720,14 @@ public <S extends SchedulableEntity> OrderingPolicy<S>
getAppOrderingPolicy(
Map<String, String> config = new HashMap<String, String>();
String confPrefix = getQueuePrefix(queue) + ORDERING_POLICY + ".";
- for (Map.Entry<String, String> kv : this) {
- if (kv.getKey().startsWith(confPrefix)) {
- config.put(kv.getKey().substring(confPrefix.length()), kv.getValue());
+ Properties props = getProps();
+ synchronized (props) {
+ for (Map.Entry<Object, Object> kv : props.entrySet()) {
Review Comment:
Yes, this only removes the per-queue copy; each leaf queue still scans all
properties, so it is still O(Q·N) with a smaller constant.
I'll add the numbers to the JIRA. With 1000 queues (medians of 3 interleaved
forks), trunk vs. trunk with this patch: load 167 → 124 ms, refresh 227 → 162
ms.
And yes, the resolver series will remove the remaining scan: from YARN-12016
on, leaf queues get their ordering policy parameters from the resolver, which
reads them through a prefix index built once per configuration
(ConfigSnapshot). I just didn't want to create the PR yet, as it would contain
this commit as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
