[ https://issues.apache.org/jira/browse/YARN-5881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15985661#comment-15985661 ]
Wangda Tan commented on YARN-5881: ---------------------------------- Thanks [~sunilg] for the POC patch, general approach looks good, several structural/cosmetic suggestions: 1) For the quotas (Like effective min/max, configured min/max, etc.) I think it is better put all of then to one class. Because it can handle following things better: a. for not existed label, it can return a zero resource instead of null. b. can have a separate lock. What I'm thinking now is moving common methods / fields from ResourceUsage to a common class, and leave all type-specifc methods (such as setCachedUsed) in the subclass. And we can create a a new class (Such as QueueResourceQuotas), which extend the base class, but with some methods to read/write queue resource quota types. Downside of the approach is, resources associated to one partition is determined by ResourceUsage#ResourceType. Without lots of changes, for each partition, it will include many unnecessary resource types (for example used_resource is unnecessary for ResourceQuotas). It looks not a big problem to me since accessing items in an array is very fast and number of partitions is limited. 2) Can we move {{loadResourceConstraintByLabelsFromConf}} from {{CSQueueUtils}} to AbstractCSQueue? Since it is more related to the queue and its parent queue, with this change lots of input parameters of {{loadResourceConstraintByLabelsFromConf}} can be removed. 3) For set effective min/max Resource, do you think we can only do it inside {{LeafQueue/ParentQueue#updateClusterResource}}? Now {{computeEffectiveResourcesForAllQueues}} creates a separate recursive call, I think it will be better to put all queue-resource-related changes inside {{LeafQueue/ParentQueue#updateClusterResource}}. Please let me know your thoughts. And for the next patch, it's better if you can update all scheduler paths to use getEffective*. > Enable configuration of queue capacity in terms of absolute resources > --------------------------------------------------------------------- > > Key: YARN-5881 > URL: https://issues.apache.org/jira/browse/YARN-5881 > Project: Hadoop YARN > Issue Type: Improvement > Reporter: Sean Po > Assignee: Wangda Tan > Attachments: > YARN-5881.Support.Absolute.Min.Max.Resource.In.Capacity.Scheduler.design-doc.v1.pdf, > YARN-5881.v0.patch, YARN-5881.v1.patch > > > Currently, Yarn RM supports the configuration of queue capacity in terms of a > proportion to cluster capacity. In the context of Yarn being used as a public > cloud service, it makes more sense if queues can be configured absolutely. > This will allow administrators to set usage limits more concretely and > simplify customer expectations for cluster allocation. -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org