[ https://issues.apache.org/jira/browse/YARN-10506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17264855#comment-17264855 ]
Gergely Pollak commented on YARN-10506: --------------------------------------- Hi [~gandras] [~wangda], *1) How we deal with "create" flag of ApplicationPlacementContext?* ** I don't think we need this flag at all. The CapacityScheduler has the information where can be dynamic queues created, it is supplied via the configuration option {{queue-path.auto-queue-creation-v2.enabled}} and {{queue-path.auto-queue-creation.enabled }} So at any point the CS will be aware if a supplied queue path is creatable. Mapping rules will return a path based on their configuration, but I don't think the placement rule should be overriding the CS configuration. This can cause some undesired issues, and we really don't get anything with it. It will be a little more inconvenient for the user since they have to configure the mapping rules AND the CS queues properly, but it doesn't look like a big deal. On the other hand there are multiple places in the code which check if a queue can be created and all these places use the queue hierarchy to determine whether a queue is a managed parent, or not, however if the Mapping Rules are allowed to circumvent this, it might lead to unforeseen inconsistencies. (As a matter of fact the CSMappingPlacementRule currently uses the information from CS to determine if a queue can be created or not, so it wouldn't set any of these flags differently then CS is configured anyway) So I would say let's omit the creation flags from the ApoplicationPlacementContext, let the CSMappingPlacementRule rely on the information provided by the CS about queue creation eligibility, and let it pass back a "regular" ApplicationPlacementContext with the desired queue path, then the CS will be able to create it if the configuration allows. This is a much easier and stable solution, while we don't really put extra administration overhead to the user, and also can be easily extended in the future. As I see circumventing protections are never a good idea even if we are talking about internal interfaces, and these flags would exactly do that, they would force the CS to create queues which are against it's configuration. > Update queue creation logic to use weight mode and allow the flexible > static/dynamic creation > --------------------------------------------------------------------------------------------- > > Key: YARN-10506 > URL: https://issues.apache.org/jira/browse/YARN-10506 > Project: Hadoop YARN > Issue Type: Sub-task > Reporter: Benjamin Teke > Assignee: Andras Gyori > Priority: Major > Attachments: YARN-10506-006-10504-010.patch, > YARN-10506-007-10504-010.patch, YARN-10506-008.patch, YARN-10506-010.patch, > YARN-10506-012.patch, YARN-10506.001.patch, YARN-10506.002.patch, > YARN-10506.003.patch, YARN-10506.004.patch, YARN-10506.005.patch, > YARN-10506.006-combined.patch, YARN-10506.006.patch, YARN-10506.007.patch, > YARN-10506.009.patch, YARN-10506.011.patch > > > The queue creation logic should be updated to use weight mode and support the > flexible creation. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org