[ https://issues.apache.org/jira/browse/YARN-10033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Payne updated YARN-10033: ------------------------------ Attachment: YARN-10033.002.patch > TestProportionalCapacityPreemptionPolicy not initializing vcores for > effective max resources > -------------------------------------------------------------------------------------------- > > Key: YARN-10033 > URL: https://issues.apache.org/jira/browse/YARN-10033 > Project: Hadoop YARN > Issue Type: Improvement > Components: capacity scheduler, test > Affects Versions: 3.3.0 > Reporter: Eric Payne > Assignee: Eric Payne > Priority: Major > Attachments: YARN-10033.001.patch, YARN-10033.002.patch > > > TestProportionalCapacityPreemptionPolicy#testPreemptionWithVCoreResource is > preempting more containers than would happen on a real cluster. > This is because the process for mocking CS queues in > {{TestProportionalCapacityPreemptionPolicy}} fails to take into consideration > vcores when mocking effective max resources. > This causes miscalculations for how many vcores to preempt when the DRF is > being used in the test: > {code:title=TempQueuePerPartition#offer} > Resource absMaxCapIdealAssignedDelta = Resources.componentwiseMax( > Resources.subtract(getMax(), idealAssigned), > Resource.newInstance(0, 0)); > {code} > In the above code, the preemption policy is offering resources to an > underserved queue. {{getMax()}} will use the effective max resource if it > exists. Since this test is mocking effective max resources, it will return > that value. However, since the mock doesn't include vcores, the test treats > memory as the dominant resource and awards too many preempted containers to > the underserved queue. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org