[ https://issues.apache.org/jira/browse/YARN-10564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17316213#comment-17316213 ]
Peter Bacsko edited comment on YARN-10564 at 4/7/21, 11:51 AM: --------------------------------------------------------------- [~gandras] thanks for the patch. >From coding POV it looks ok, this is more like a high level review. There's are some things I just can't figure out (maybe I'm in a bad shape today). 1. Let's say you set the capacity 6w for {{root.a.*}}. Then a dynamic queue {{root.a.newparent.newchild}} get created. How does the weight settings propagate to "newparent" and "newchild"? I kept looking at the code, but it's just not obvious. I can see that "root.a" will have an entry in {{templateEntries}}, but then what? 2. I can't deciper this part: {noformat} for (int i = 0; i <= wildcardLevel; ++i) { queuePathParts.set(queuePathParts.size() - 1 - i, WILDCARD_QUEUE); } {noformat} What's happening here? 3. There is a variable called "supportedWildcardLevel". What is "supported" means in this context? Later on we set it to {{Math.min(queueHierarchyParts - 1, MAX_WILDCARD_LEVEL);}}. It seems to me that it is either 0 or 1, because {{MAX_WILDCARD_LEVEL}} is 1. I assume most of the time it's going to be 1? I don't understand what it is meant to represent. was (Author: pbacsko): [~gandras] thanks for the patch. >From coding POV it looks ok, this is more like a high level review. There's are some things I just can't figure out (maybe I'm in a bad shape today). 1. Let's say you set the capacity 6w for {{root.a.*}}. Then a dynamic queue {{root.a.newparent.newchild}} get created. How does the weight settings propagate to "newparent" and "newchild"? I kept looking at the code, but it's just not obvious. I can see that "root.a" will have an entry in {{templateEntries}}, but then what? 2. I can't deciper this part: {noformat} for (int i = 0; i <= wildcardLevel; ++i) { queuePathParts.set(queuePathParts.size() - 1 - i, WILDCARD_QUEUE); } {noformat} What's happening here? 3. There is a variable called "supportedWildcardLevel". What is "supported" means in this context? Later on we set it to {{Math.min(queueHierarchyParts - 1, MAX_WILDCARD_LEVEL);}}. It seems to me that it is either 0 or 1, because {{MAX_WILDCARD_LEVEL}} is 1. I assume most of the time it's going to be 1? Mentally I don't understand what it is meant to represent. > Support Auto Queue Creation template configurations > --------------------------------------------------- > > Key: YARN-10564 > URL: https://issues.apache.org/jira/browse/YARN-10564 > Project: Hadoop YARN > Issue Type: Sub-task > Reporter: Andras Gyori > Assignee: Andras Gyori > Priority: Major > Attachments: YARN-10564.001.patch, YARN-10564.002.patch, > YARN-10564.003.patch, YARN-10564.004.patch, YARN-10564.005.patch, > YARN-10564.poc.001.patch > > > Similar to how the template configuration works for ManagedParents, we need > to support templates for the new auto queue creation logic. Proposition is to > allow wildcards in template configs such as: > {noformat} > yarn.scheduler.capacity.root.*.*.weight 10{noformat} > which would mean, that set weight to 10 of every leaf of every parent under > root. > We should possibly take an approach, that could support arbitrary depth of > template configuration, because we might need to lift the limitation of auto > queue nesting. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org