[
https://issues.apache.org/jira/browse/YARN-796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068169#comment-14068169
]
Allen Wittenauer commented on YARN-796:
---------------------------------------
bq. An important use case we can see is, some customers want to mark label on
each node indicate which department/team the node belongs to, when a new team
comes in and new machines added, labels may need to be changed.
You can solve this problem today by just running separate RMs. In practice,
however, marking nodes for specific teams in queue systems doesn't work because
doing so assumes that the capacity never changes... i.e., nodes never fail.
That happens all the time, of course, thus why percentages make a lot more
sense. If you absolutely want a fixed number of capacity, you still wouldn't
mark specific nodes: you'd say "queue x gets y machines" with no specification
of which nodes.
bq. And also, it is possible that the whole cluster is booked to run some huge
batch job at 12am-2am for example. So such labels will be changed frequently.
Well, no, they won't. They'll happen exactly twice a day. But it doesn't
matter: you can solve this problem today too by just setting something that
changes the queue acls at 12am and 2am via a cron job.
bq. For example user may want to run HBase master/region-servers in a group of
nodes, and don't want any other tasks running in these nodes even if they have
free resource. Our current queue configuration cannot solve such problem
... except, you guessed it: this is a solved problem today too. You just need
to make sure the container sizes that are requested consume the whole node.
bq. If we only have distributed configuration on each node, it is a nightmare
for admins to re-configure.
Hi. My name is Allen and I'm an admin. Even if using labels for doing this
type of scheduling was sane, it still wouldn't be a nightmare because any
competent admin would use configuration management to roll out changes to the
nodes in a controlled manner.
But more importantly: these use cases are *solved problems* and have been in
YARN for a very long time.
> Allow for (admin) labels on nodes and resource-requests
> -------------------------------------------------------
>
> Key: YARN-796
> URL: https://issues.apache.org/jira/browse/YARN-796
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Arun C Murthy
> Assignee: Wangda Tan
> Attachments: LabelBasedScheduling.pdf,
> Node-labels-Requirements-Design-doc-V1.pdf, YARN-796.patch
>
>
> It will be useful for admins to specify labels for nodes. Examples of labels
> are OS, processor architecture etc.
> We should expose these labels and allow applications to specify labels on
> resource-requests.
> Obviously we need to support admin operations on adding/removing node labels.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
