[
https://issues.apache.org/jira/browse/YARN-10506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17265144#comment-17265144
]
Peter Bacsko commented on YARN-10506:
-------------------------------------
"_It is possible when the rule checks, the queue allow creation, but when
ApplicationPlacementContext arrived at scheduler, the queue is refreshed"_
This indeed sounds like a race condition, but if the queue is removed in the
meantime, then the submission would fail anyway. The refresh itself is perfomed
while holding a write lock, so that in itself should not be a problem.
"RuleEngine can just pass ApplicationPLacementContext to Scheduler and let
scheduler making the decision"
Could you elaborate on what "decision making" you're referring to? I think
inside the engine, it's important that it has to determine whether the target
queue is available or not, if not, can we create it, etc. The logic must be
there, otherwise we can't proceed to the next rule if the fallback is set to
"skip". If you perform the validations (which are currently done by
{{CSMappingPlacementRule}}) outside of the placement engine, you lose the
ability to execute different actions (skip/reject/place to default).
> Update queue creation logic to use weight mode and allow the flexible
> static/dynamic creation
> ---------------------------------------------------------------------------------------------
>
> Key: YARN-10506
> URL: https://issues.apache.org/jira/browse/YARN-10506
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Benjamin Teke
> Assignee: Andras Gyori
> Priority: Major
> Attachments: YARN-10506-006-10504-010.patch,
> YARN-10506-007-10504-010.patch, YARN-10506-008.patch, YARN-10506-010.patch,
> YARN-10506-012.patch, YARN-10506.001.patch, YARN-10506.002.patch,
> YARN-10506.003.patch, YARN-10506.004.patch, YARN-10506.005.patch,
> YARN-10506.006-combined.patch, YARN-10506.006.patch, YARN-10506.007.patch,
> YARN-10506.009.patch, YARN-10506.011.patch
>
>
> The queue creation logic should be updated to use weight mode and support the
> flexible creation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
