[
https://issues.apache.org/jira/browse/YARN-2154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14679355#comment-14679355
]
Ben Podgursky commented on YARN-2154:
-------------------------------------
Hi, I've been watching this ticket since this has been a big problem on our
cluster. Karthik, your last comment concerned me because it seems like it
will violate one of the things FairScheduler is supposed to promise:
"Unlike the default Hadoop scheduler, which forms a queue of jobs, this lets
short jobs finish in reasonable time while not starving long jobs.'
and
"When other jobs are submitted, tasks slots that free up are assigned to the
new jobs, so that each job gets roughly the same amount of CPU time. "
If a job can't even get an AM, it surely won't be able to get an equitable
portion of slots for Maps and Reduces -- being able to get small jobs through
quickly even when the pool has huge jobs running is the reason we use
FairScheduler.
Sorry if I'm misunderstanding that comment.
> FairScheduler: Improve preemption to preempt only those containers that would
> satisfy the incoming request
> ----------------------------------------------------------------------------------------------------------
>
> Key: YARN-2154
> URL: https://issues.apache.org/jira/browse/YARN-2154
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: fairscheduler
> Affects Versions: 2.4.0
> Reporter: Karthik Kambatla
> Assignee: Arun Suresh
> Priority: Critical
> Attachments: YARN-2154.1.patch
>
>
> Today, FairScheduler uses a spray-gun approach to preemption. Instead, it
> should only preempt resources that would satisfy the incoming request.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
