[
https://issues.apache.org/jira/browse/YARN-9879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17011993#comment-17011993
]
Gergely Pollak commented on YARN-9879:
--------------------------------------
Thank you for your feedbacks [~wilfreds], [~pbacsko], [~leftnoteasy]!
{quote}The important part is applying a new configuration. If the configuration
adds a leaf queue that is not unique the configuration update currently is
rejected. With this change we would allow that config to become active. This
*could* break existing applications when they try to submit to the leaf queue
that is no longer unique.
{quote}
I think this can be a major issue. In a scenario when the user have let's say a
job which runs daily, and it's been working like that for years, can simply
break, because an other user, totally unrelated to his team or even department
creates a queue with the same name. The suddenly this application will start
failing, and even if we add logs which explicitly tell what is the reason
behind the application rejection, user one might not even notice it started
failing. (We shouldn't assume everyone has proper monitoring and warning
systems). So technically any user who can create a queue, can disable an other
user's application if it is started by using single queue name reference. And
it concerns me a bit.
However [~adam.antal] had an idea to even fix this issue, and then we can move
forwards with Wilfred's suggestion:
We should make it possible to flag QUEUES for using full queue name reference,
and these flags should mean ALL queues under the flagged queue can ONLY be
referenced by full queue name. For regular queues (non-flagged ones), we would
still enforce the unique leaf queue policy, while newer or migrating users
could stick to the full queue reference. This proposal can also helps gradual
migration, for older CS users buy slowly flagging queues in which the
applications are already started by queue name.
What do you think of this idea?
> Allow multiple leaf queues with the same name in CS
> ---------------------------------------------------
>
> Key: YARN-9879
> URL: https://issues.apache.org/jira/browse/YARN-9879
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Gergely Pollak
> Assignee: Gergely Pollak
> Priority: Major
> Attachments: DesignDoc_v1.pdf
>
>
> Currently the leaf queue's name must be unique regardless of its position in
> the queue hierarchy.
> Design doc and first proposal is being made, I'll attach it as soon as it's
> done.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
