[ https://issues.apache.org/jira/browse/YARN-10504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17262301#comment-17262301 ]
Wangda Tan commented on YARN-10504: ----------------------------------- Still see some issues around auto created leaf queue, fixed a number of issues introduced by the latest changes in ParentQueue.setChildQueues. Unit tests: - TestRMWebServicesForCSWithPartitions: [FIXED] There's bad assumption for #resourceByPartition returned. When the queue is initialized, there's no usage thus there should have no resourceByPartition. Updated tests to not test #resourceByPartition. - TestRMWebServices: [FIXED] testValidateAndGetSchedulerConfigurationInvalidConfig: This is a bad test which checks returned plain text of exception, we should avoid do that. fix: Changed to just check it is an IOException instead of checking the actual text. * TestAbsoluteResourceConfiguration [PENDING] testSimpleMinMaxResourceConfigurartionPerQueue: this is still pending, because now AutoCreatedLeafQueue depends on mergeCapacities, which calculate capacity first, then use the capacity to calculate absolute min resource. It can cause a change in effectiveMinResource value even if cluster has sufficient resource. We should look into this once we finish the YARN-10506. Now temporarily update test to use the wrong value to unblock tests. fixed rest issues. Will continue investigate. > Implement weight mode in Capacity Scheduler > ------------------------------------------- > > Key: YARN-10504 > URL: https://issues.apache.org/jira/browse/YARN-10504 > Project: Hadoop YARN > Issue Type: Sub-task > Reporter: Benjamin Teke > Assignee: Benjamin Teke > Priority: Major > Attachments: YARN-10504.001.patch, YARN-10504.002.patch, > YARN-10504.003.patch, YARN-10504.004.patch, YARN-10504.005.patch, > YARN-10504.006.patch, YARN-10504.007.patch, YARN-10504.008.patch, > YARN-10504.ver-1.patch, YARN-10504.ver-2.patch, YARN-10504.ver-3.patch > > > To allow the possibility to flexibly create queues in Capacity Scheduler a > weight mode should be introduced. The existing \{{capacity }}property should > be used with a different syntax, i.e: > root.users.capacity = (1.0) or ~1.0 or ^1.0 or @1.0 > root.users.capacity = 1.0w > root.users.capacity = w:1.0 > Weight support should not impact the existing functionality. > > The new functionality should: > * accept and validate the new weight values > * enforce a singular mode on the whole queue tree > * (re)calculate the relative (percentage-based) capacities based on the > weights during launch and every time the queue structure changes -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org