[
https://issues.apache.org/jira/browse/YARN-3409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15707465#comment-15707465
]
Konstantinos Karanasos commented on YARN-3409:
----------------------------------------------
Hi guys, thank you for driving this, for the documentation, and for all the
discussion. I think it is a super-useful feature to have.
We also built a prototype over the summer with the end goal of supporting
placement constraints (of the form affinity/anti-affinity/cardinality, similar
in spirit to YARN-1042), and had to add some initial support for similar node
labels along the way.
Please find some comments below.
# As was mentioned above in one of [~leftnoteasy]'s comments too, I also
(strongly :)) suggest to use these ConstraintNodeLabels in the context of
YARN-1042 for supporting (anti-)affinity constraints as well. I think it will
greatly avoid duplicate effort and simplify the code.
# On a similar note, can these ContraintNodeLabels be added/removed
dynamically? For example, when a container starts its execution, it might
contain some attributes (to be added -- I know such attributes cannot be
specified at the moment). Those attributes will then be added to the node's
labels, for the time the container is running. This can be useful for
(anti-)affinity constraints. For instance, a task can add the label
"HBase-master", and then another resource request can have a constraint of the
form "don't put me at a node with an HBase-master label". What do you think?
# A few people above mentioned that the naming of ContainerNodeLabels might not
be ideal. I think they look more like attributes (as in key-value pairs), so we
might consider using a name that denotes that (labels sound to me more like
something that exists or not, but does not have a value).
# I like that you don't take headroom into account when it comes to constraint
label expressions.
# +1 for Option 1. It might also be that the implementation of
ConstraintNodeLabels will be easier at some places than that of
NodeLabels/Partitions (e.g., given there is no need for supporting headroom).
In terms of logistics, +1 for the branch too. I think we should make this an
umbrella JIRA.
# Can you please give an example of a cluster-level constraint?
# bq. Constraints will be matched within the scope of a node partition.
Making sure I understand.. Why do we need this constraint? I think they are
orthogonal, right? Unless you mean that if the user specifies a constraint, it
has to be taken into account too, which I understand.
# Also adding one last thing we did in our prototype that I think is related to
the locality (node/rack/any) discussion above and might be useful to consider.
We assumed that the ConstraintNodeLabels are following the hierarchy of the
cluster. That is, a rack was inheriting the ConstraintNodeLabels of all its
nodes. A detail here is that we considered only existential
ConstraintNodeLabels (as I mentioned above, without values), which avoids
conflicts. In the more general case you are describing, it is not clear what
happens if a node of the rack has Java 6 and the other Java 7 (not clear what
should be the label of the rack). We will need to resolve conflicts in those
case. However, I think that design is quite powerful. Think that eventually we
can even define different logical classes of nodes and register them in the RM.
For instance, group nodes that belong to the same upgrade domain (being
upgraded at the same time -- we see this use case a lot in our clusters).
> Add constraint node labels
> --------------------------
>
> Key: YARN-3409
> URL: https://issues.apache.org/jira/browse/YARN-3409
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: api, capacityscheduler, client
> Reporter: Wangda Tan
> Assignee: Naganarasimha G R
> Attachments: Constraint-Node-Labels-Requirements-Design-doc_v1.pdf
>
>
> Specify only one label for each node (IAW, partition a cluster) is a way to
> determinate how resources of a special set of nodes could be shared by a
> group of entities (like teams, departments, etc.). Partitions of a cluster
> has following characteristics:
> - Cluster divided to several disjoint sub clusters.
> - ACL/priority can apply on partition (Only market team / marke team has
> priority to use the partition).
> - Percentage of capacities can apply on partition (Market team has 40%
> minimum capacity and Dev team has 60% of minimum capacity of the partition).
> Constraints are orthogonal to partition, they’re describing attributes of
> node’s hardware/software just for affinity. Some example of constraints:
> - glibc version
> - JDK version
> - Type of CPU (x86_64/i686)
> - Type of OS (windows, linux, etc.)
> With this, application can be able to ask for resource has (glibc.version >=
> 2.20 && JDK.version >= 8u20 && x86_64).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
