[ https://issues.apache.org/jira/browse/YARN-8495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16533902#comment-16533902 ]
Yufei Gu commented on YARN-8495: -------------------------------- Thanks [~Dillon.] for filing this. The implementation conflicts with the preemption inside a queue, which assumes each application splits the resource evenly, so that scheduler will kill the containers from the applications who use more resource than its fair share. In a case that a low priority app A uses less than its fair share and a high priority app B uses more than its fair share. This priority scheduling will assign containers to B, while the preemption will kill B's containers and intend to give them to A. A live lock happens. > Priority scheduling support in FairShare scheduler > -------------------------------------------------- > > Key: YARN-8495 > URL: https://issues.apache.org/jira/browse/YARN-8495 > Project: Hadoop YARN > Issue Type: Wish > Components: fairscheduler > Reporter: Dillon Zhang > Priority: Major > Attachments: YARN-8495.001.patch > > > In production environment, priority scheduling is of vital importance to us > as we have lots of queues for different departments, then create applications > but some of them are not so important as others, so we must guarantee the > import ones to supply service. > Based on the priority of the application, Fair Scheduler should be able to > give preference to application while scheduling. > Comparator<FairShareComparator> applicationComparator can be changed as > below. > 1. Check for Application priority. If priority is available, then return the > highest priority job. > 2. Otherwise continue with existing logic such as Fair Share comparison and > App ID comparison and then TimeStamp comparison. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org