[ https://issues.apache.org/jira/browse/YARN-10559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17263089#comment-17263089 ]
Sunil G edited comment on YARN-10559 at 1/12/21, 5:42 AM: ---------------------------------------------------------- [~ananyo_rao] Thanks for the efforts. Few minor comments. # Please take a look at the newly introduced 7 checkstyle warnings and help to check how many of them can be fixed. [https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/457/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt] # In setFairShareForApps, there could be some chances for divide by 0. Let me quote those here. {code:java} int numOfAppsInQueue = tq.leafQueue.getAllApplications().size(); Resource fairShareAcrossApps = Resources.divideAndCeil( this.rc, queueReassignableResource, numOfAppsInQueue); {code} We are handling this internally. but still, it's better to add a check. # It's better to ensure two points here. We should get a non-negative resource and its better to check & ensurefairShareForApp cannot be 0 {code:java} Resource fairShareForApp = Resources.min( rc, clusterResource, fairShareAcrossApps, fairShareWithinUL);{code} was (Author: sunilg): [~ananyo_rao] Thanks for the efforts. Few minor comments. # Please take a look at the newly introduced 7 checkstyle warnings and help to check how many of them can be fixed. [https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/457/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt] # In setFairShareForApps, there could be some chances for divide by 0. Let me quote those here. {code:java} int numOfAppsInQueue = tq.leafQueue.getAllApplications().size(); Resource fairShareAcrossApps = Resources.divideAndCeil( this.rc, queueReassignableResource, numOfAppsInQueue); {code} We are handling this internally. but still, it's better to add a check. # It's better to ensure two points here. We should get a non-negative resource and its better to check & ensurefairShareForApp cannot be 0 {code:java} Resource fairShareForApp = Resources.min( rc, clusterResource, fairShareAcrossApps, fairShareWithinUL);{code} > Fair sharing intra-queue preemption support in Capacity Scheduler > ----------------------------------------------------------------- > > Key: YARN-10559 > URL: https://issues.apache.org/jira/browse/YARN-10559 > Project: Hadoop YARN > Issue Type: Improvement > Components: capacityscheduler > Affects Versions: 3.1.4 > Reporter: VADAGA ANANYO RAO > Assignee: VADAGA ANANYO RAO > Priority: Major > Attachments: FairOP_preemption-design_doc_v1.pdf, > FairOP_preemption-design_doc_v2.pdf, YARN-10559.0001.patch, > YARN-10559.0002.patch, YARN-10559.0003.patch, YARN-10559.0004.patch > > Original Estimate: 168h > Remaining Estimate: 168h > > Usecase: > Due to the way Capacity Scheduler preemption works, If a single user submits > a large application to a queue (using 100% of resources), that job will not > be preempted by future applications from the same user within the same queue. > This implies that the later applications will be forced to wait for > completion of the long running application. This prevents multiple long > running, large, applications from running concurrently. > Support fair sharing among apps while preempting applications from same queue. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org