[
https://issues.apache.org/jira/browse/YARN-10506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17263999#comment-17263999
]
Andras Gyori edited comment on YARN-10506 at 1/13/21, 9:07 AM:
---------------------------------------------------------------
Thank you [~wangda] for the review. My comments:
# My initial approach was exactly this, however, did not want to extend the
scope of this patch. Will refactor the mentioned parts in a followup jira.
# This was the approach at the beginning, however, we talked about this with
[~shuzirra], and came to the conclusion, that:
## The auto queue creation should be used when mapping rules are turned off,
therefore placement context driven AQC would be a limitation
We also concluded, that its not the responsibility of the mapping
rule/PlacementContext to decide whether AQC is enabled or not, the create flag
will only be used to do a preliminary check of the possibility
The ParentQueue isEligibleForAutoQueueCreation method was made to support this
mapping rule check
## I will check it out, thank you for noticing!
## Edit: [~zhuqi]'s solution is good, I approve.
# We have argued about this problem, and I agree, that the auto queue/dynamic
queue terminology is overused, and wanted to make a proposal, to distinguish
this new AQC from the ManagedParent old logic. However, I think it is still a
good idea to use a property to drive this behaviour due to security reasons: if
a customer wants to use weight mode, the AQC will implicitly be turned on, and
it might be not desirable (rogue users could create queues anywhere). I prefer
explicit allowance in this case. That said, we should come up with a
distinguishable terminology for the new auto queue creation (perhaps saying
auto-queue-creation-v2 or extended).
# I too found this problematic, and will address this issue.
# I will add the testcase.
was (Author: gandras):
Thank you [~wangda] for the review. My comments:
# My initial approach was exactly this, however, did not want to extend the
scope of this patch. Will refactor the mentioned parts in a followup jira.
# This was the approach at the beginning, however, we talked about this with
[~shuzirra], and came to the conclusion, that:
## The auto queue creation should be used when mapping rules are turned off,
therefore placement context driven AQC would be a limitation
We also concluded, that its not the responsibility of the mapping
rule/PlacementContext to decide whether AQC is enabled or not, the create flag
will only be used to do a preliminary check of the possibility
The ParentQueue isEligibleForAutoQueueCreation method was made to support this
mapping rule check
## I will check it out, thank you for noticing!
## I am not sure what you mean by that, but I will check it out. However, I
have distinguished the parent hierarchy in its own method to be more flexible
(we might need to support deeper hierarchy, more rigorous checks etc.)
# We have argued about this problem, and I agree, that the auto queue/dynamic
queue terminology is overused, and wanted to make a proposal, to distinguish
this new AQC from the ManagedParent old logic. However, I think it is still a
good idea to use a property to drive this behaviour due to security reasons: if
a customer wants to use weight mode, the AQC will implicitly be turned on, and
it might be not desirable (rogue users could create queues anywhere). I prefer
explicit allowance in this case. That said, we should come up with a
distinguishable terminology for the new auto queue creation (perhaps saying
auto-queue-creation-v2 or extended).
# I too found this problematic, and will address this issue.
# I will add the testcase.
> Update queue creation logic to use weight mode and allow the flexible
> static/dynamic creation
> ---------------------------------------------------------------------------------------------
>
> Key: YARN-10506
> URL: https://issues.apache.org/jira/browse/YARN-10506
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Benjamin Teke
> Assignee: Andras Gyori
> Priority: Major
> Attachments: YARN-10506-006-10504-010.patch,
> YARN-10506-007-10504-010.patch, YARN-10506-008.patch, YARN-10506-010.patch,
> YARN-10506.001.patch, YARN-10506.002.patch, YARN-10506.003.patch,
> YARN-10506.004.patch, YARN-10506.005.patch, YARN-10506.006-combined.patch,
> YARN-10506.006.patch, YARN-10506.007.patch, YARN-10506.009.patch
>
>
> The queue creation logic should be updated to use weight mode and support the
> flexible creation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
