[ https://issues.apache.org/jira/browse/YARN-6151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15854730#comment-15854730 ]
Yufei Gu edited comment on YARN-6151 at 2/6/17 9:14 PM: -------------------------------------------------------- Uploaded one simple solution which is before we decide which queue to preempt, we check if this queue can be preempted. For leaf queue, we check if the queue uses more resources than its fair share. For parent queue, we need to recursively go down to each level to check. There might be a performance concern, but considering the performance is still acceptable before YARN-5969 which did recursively get resource usage anyway, it should be fine. was (Author: yufeigu): Uploaded one simple solution is that before we choose which queue to preempt, we check if this queue can be preempted. For leaf queue, we just check if the queue uses more resources than its fair share. For parent queue, we need to recursively go down to each level to check. There might be a performance concern, but considering the performance is still acceptable before YARN-5969 which did recursively get resource usage anyway, it should be fine. > FS Preemption doesn't filter out queues which cannot be preempted > ----------------------------------------------------------------- > > Key: YARN-6151 > URL: https://issues.apache.org/jira/browse/YARN-6151 > Project: Hadoop YARN > Issue Type: Bug > Components: fairscheduler > Affects Versions: 2.8.0 > Reporter: Yufei Gu > Assignee: Yufei Gu > Attachments: YARN-6151.001.patch > > > This is preemption bug happens before 2.8.0, which also described in > YARN-3405. > Queue hierarchy described as below: > {noformat} > root > / \ > queue-1 queue-2 > / \ > queue-1-1 queue-1-2 > {noformat} > Assume cluster resource is 100 and all queues have same weights. > # queue-1-1 and queue-2 has apps. Each get 50 usage and 50 fairshare. > # When queue-1-2 is active, supposedly it will preempt 25 from queue-1-1, but > this doesn't happen because preemption happens top-down, queue-2 could be the > preemption candidate as long as queue-2 is less needy than queue-1, and > queue-2 doesn't exceed the fair share which means preemption won't happen. > We need to filter out queue-2 since it isn't a valid candidate. -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org