[ https://issues.apache.org/jira/browse/YARN-10564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17316262#comment-17316262 ]
Andras Gyori commented on YARN-10564: ------------------------------------- [~pbacsko] thank you for your review. I was afraid that the will be a little bit convoluted. This was much worse in case of legacy auto created queues. Storing the children templates on the Parent itself might be more clear, I will try to investigate that approach. What is happening in this patch: # The template configuration class sets the properties on the queue itself # Since we will support configurable depth of queue creation, we need to support variable length of wildcarding as well (so in case of QUEUE_CREATION_DEPTH=3, we need to support 2 wildcard levels eg. root.a.*.*, thats why we have MAX_WILDCARD_LEVEL) # For a dynamic LeafQueue *root.a.a1,* we start looking for template entries for its parent *root.a* in the configuration and increasing the wildcard level by 1 each iteration. An example for the iteration: ## root.a.auto-queue-creation-v2.template.weight ## root.*.auto-queue-creation-v2.template.weight # After this, we cut the _auto-queue-creation-v2.template_ prefix, and we set the actual value for the configuration. An example: ## from root.a.auto-queue-creation-v2.template.weight -> root.a.a1.weight # If we set the root.a.a1.weight explicitly (this would be used for STOPPING / RUNNING the queue), a template configuration would not overwrite the value. An example: ## root.a.auto-queue-creation-v2.template.weight = 2 ## root.a.a1.weight = 3 ## root.a.a1 will have weight = 3 I hope it does clarify the patch to some extent. > Support Auto Queue Creation template configurations > --------------------------------------------------- > > Key: YARN-10564 > URL: https://issues.apache.org/jira/browse/YARN-10564 > Project: Hadoop YARN > Issue Type: Sub-task > Reporter: Andras Gyori > Assignee: Andras Gyori > Priority: Major > Attachments: YARN-10564.001.patch, YARN-10564.002.patch, > YARN-10564.003.patch, YARN-10564.004.patch, YARN-10564.005.patch, > YARN-10564.poc.001.patch > > > Similar to how the template configuration works for ManagedParents, we need > to support templates for the new auto queue creation logic. Proposition is to > allow wildcards in template configs such as: > {noformat} > yarn.scheduler.capacity.root.*.*.weight 10{noformat} > which would mean, that set weight to 10 of every leaf of every parent under > root. > We should possibly take an approach, that could support arbitrary depth of > template configuration, because we might need to lift the limitation of auto > queue nesting. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org