[
https://issues.apache.org/jira/browse/YARN-10506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17265085#comment-17265085
]
Wangda Tan commented on YARN-10506:
-----------------------------------
Thanks [~shuzirra], [~pbacsko], posting this comment while in the middle of
something else, I hope I didn't miss important information.
Here you shared several things:
*1) Add a create flag on the rule itself in the JSON:*
I agreed.
*2) yarh.scheduler.capacity.<path>.auto-queue-creation-v2.enabled*
I agreed.
Also, I agree we should look at if we can make the property inheritable or not,
I agree but I suggest to move the implementation to a separate Jira.
*3) Removing "create" from the ApplicationPlacementContext:*
I initially thought it is needed because the rule engine may not do exhaust
checks and the concurrency issue. _(It is possible when the rule checks, the
queue allow creation, but when ApplicationPlacementContext arrived at
scheduler, the queue is refreshed)_. So I think it is still valuable to let the
scheduler do the creation based on demand, and fail the App submission
atomically.
Also, if we relies on Scheduler to check creatable or not, it can reduce
complexities of RuleEngine to do additional check. RuleEngine can just pass
ApplicationPLacementContext to Scheduler and let scheduler making the decision.
> Update queue creation logic to use weight mode and allow the flexible
> static/dynamic creation
> ---------------------------------------------------------------------------------------------
>
> Key: YARN-10506
> URL: https://issues.apache.org/jira/browse/YARN-10506
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Benjamin Teke
> Assignee: Andras Gyori
> Priority: Major
> Attachments: YARN-10506-006-10504-010.patch,
> YARN-10506-007-10504-010.patch, YARN-10506-008.patch, YARN-10506-010.patch,
> YARN-10506-012.patch, YARN-10506.001.patch, YARN-10506.002.patch,
> YARN-10506.003.patch, YARN-10506.004.patch, YARN-10506.005.patch,
> YARN-10506.006-combined.patch, YARN-10506.006.patch, YARN-10506.007.patch,
> YARN-10506.009.patch, YARN-10506.011.patch
>
>
> The queue creation logic should be updated to use weight mode and support the
> flexible creation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
