[
https://issues.apache.org/jira/browse/YARN-10807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shilun Fan updated YARN-10807:
------------------------------
Component/s: capacity scheduler
Hadoop Flags: Reviewed
Target Version/s: 3.4.0
Affects Version/s: 3.4.0
> Parents node labels are incorrectly added to child queues in weight mode
> -------------------------------------------------------------------------
>
> Key: YARN-10807
> URL: https://issues.apache.org/jira/browse/YARN-10807
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacity scheduler
> Affects Versions: 3.4.0
> Reporter: Benjamin Teke
> Assignee: Benjamin Teke
> Priority: Major
> Fix For: 3.4.0
>
> Attachments: YARN-10807.001.patch, YARN-10807.002.patch
>
>
> In ParentQueue.updateClusterResource when calculating the normalized weights
> CS will iterate through the parent's nodelabels.
> If the parent has a node label that a specific child doesn't, it will
> incorrectly add it to the child's node label list through the
> queueCapacities.setNormalizedWeights(label, weight) call:
> {code:java}
> // Normalize weight of children
> if (getCapacityConfigurationTypeForQueues(childQueues)
> == QueueCapacityType.WEIGHT) {
> for (String nodeLabel : queueCapacities.getExistingNodeLabels()) {
> float sumOfWeight = 0;
> for (CSQueue queue : childQueues) {
> float weight = Math.max(0,
> queue.getQueueCapacities().getWeight(nodeLabel));
> sumOfWeight += weight;
> }
> // When sum of weight == 0, skip setting normalized_weight (so
> // normalized weight will be 0).
> if (Math.abs(sumOfWeight) > 1e-6) {
> for (CSQueue queue : childQueues) {
> queue.getQueueCapacities().setNormalizedWeight(nodeLabel,
> queue.getQueueCapacities().getWeight(nodeLabel) /
> sumOfWeight);
> }
> }
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
