[ https://issues.apache.org/jira/browse/YARN-3409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16096324#comment-16096324 ]
Sunil G commented on YARN-3409: ------------------------------- Hi [~templedf] Lemme take the liberty of pointing out some thoughts here. Currently we have CommonNodeLabelManager which handles node label and other common tasks such as # Api Handling # Storage and mappings # Recovery For internal scheduler access point, a derivative version of RMNodeLabelManager is used. All together, as you mentioned we might need to reinvent whole wheel here. Ideally we would be looking a Common base class to handle recovery, storage and basic mappings. NodeLabel specific manager could derive from this base class to handle internal node label specific apis etc. Similarly for attributes, we can have a RMNodeAttribute manage derived from the common class. In a nutshell, there are some pieces in node label and attributes which are different and it could be handled respectively. About the point +"value for the label could change between when the allocation is made and when the AM launches the container"+ , I think it is kind of handled now in node label. We could see something like containers which are scheduler based on old attribute and still running, and new set of containers which are scheduled based on new attribute. If container demand for new attribute is more on that node, all old style containers could be marked for preemption. In fact, we do same now for node label (when partition of node is changed runtime). Is this a possible way of handling same. Thoughts? I think [~naganarasimha...@apache.org] also has more thoughts on same. Please correct me if I am wrong. Thank You. > Add constraint node labels > -------------------------- > > Key: YARN-3409 > URL: https://issues.apache.org/jira/browse/YARN-3409 > Project: Hadoop YARN > Issue Type: Sub-task > Components: api, capacityscheduler, client > Reporter: Wangda Tan > Assignee: Naganarasimha G R > Attachments: Constraint-Node-Labels-Requirements-Design-doc_v1.pdf, > YARN-3409.WIP.001.patch > > > Specify only one label for each node (IAW, partition a cluster) is a way to > determinate how resources of a special set of nodes could be shared by a > group of entities (like teams, departments, etc.). Partitions of a cluster > has following characteristics: > - Cluster divided to several disjoint sub clusters. > - ACL/priority can apply on partition (Only market team / marke team has > priority to use the partition). > - Percentage of capacities can apply on partition (Market team has 40% > minimum capacity and Dev team has 60% of minimum capacity of the partition). > Constraints are orthogonal to partition, they’re describing attributes of > node’s hardware/software just for affinity. Some example of constraints: > - glibc version > - JDK version > - Type of CPU (x86_64/i686) > - Type of OS (windows, linux, etc.) > With this, application can be able to ask for resource has (glibc.version >= > 2.20 && JDK.version >= 8u20 && x86_64). -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org