[
https://issues.apache.org/jira/browse/MAPREDUCE-706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12743734#action_12743734
]
Matei Zaharia commented on MAPREDUCE-706:
-----------------------------------------
I've created MAPREDUCE-878 to rename the design doc and add the license header.
With the PDF, I'm worried that many people won't have LaTeX installed, and it
will be difficult to write an ant task to compile it, but if having a PDF in
there is a problem, I can remove it. It's only 120 KB in size though.
> Support for FIFO pools in the fair scheduler
> --------------------------------------------
>
> Key: MAPREDUCE-706
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-706
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Components: contrib/fair-share
> Reporter: Matei Zaharia
> Assignee: Matei Zaharia
> Fix For: 0.21.0
>
> Attachments: fsdesigndoc.pdf, fsdesigndoc.tex, mapreduce-706.patch,
> mapreduce-706.v1.patch, mapreduce-706.v2.patch, mapreduce-706.v3.patch,
> mapreduce-706.v4.patch, mapreduce-706.v5.patch
>
>
> The fair scheduler should support making the internal scheduling algorithm
> for some pools be FIFO instead of fair sharing in order to work better for
> batch workloads. FIFO pools will behave exactly like the current default
> scheduler, sorting jobs by priority and then submission time. Pools will have
> their scheduling algorithm set through the pools config file, and it will be
> changeable at runtime.
> To support this feature, I'm also changing the internal logic of the fair
> scheduler to no longer use deficits. Instead, for fair sharing, we will
> assign tasks to the job farthest below its share as a ratio of its share.
> This is easier to combine with other scheduling algorithms and leads to a
> more stable sharing situation, avoiding unfairness issues brought up in
> MAPREDUCE-543 and MAPREDUCE-544 that happen when some jobs have long tasks.
> The new preemption (MAPREDUCE-551) will ensure that critical jobs can gain
> their fair share within a bounded amount of time.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
