[ https://issues.apache.org/jira/browse/MAPREDUCE-706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12743759#action_12743759 ]
Tsz Wo (Nicholas), SZE commented on MAPREDUCE-706: -------------------------------------------------- > ... With the PDF, I'm worried that many people won't have LaTeX installed, > and it will be difficult to write an ant task to compile it, ... I agree. Let's leave the pdf file there for the moment. We may come up a solution later. > Support for FIFO pools in the fair scheduler > -------------------------------------------- > > Key: MAPREDUCE-706 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-706 > Project: Hadoop Map/Reduce > Issue Type: New Feature > Components: contrib/fair-share > Reporter: Matei Zaharia > Assignee: Matei Zaharia > Fix For: 0.21.0 > > Attachments: fsdesigndoc.pdf, fsdesigndoc.tex, mapreduce-706.patch, > mapreduce-706.v1.patch, mapreduce-706.v2.patch, mapreduce-706.v3.patch, > mapreduce-706.v4.patch, mapreduce-706.v5.patch > > > The fair scheduler should support making the internal scheduling algorithm > for some pools be FIFO instead of fair sharing in order to work better for > batch workloads. FIFO pools will behave exactly like the current default > scheduler, sorting jobs by priority and then submission time. Pools will have > their scheduling algorithm set through the pools config file, and it will be > changeable at runtime. > To support this feature, I'm also changing the internal logic of the fair > scheduler to no longer use deficits. Instead, for fair sharing, we will > assign tasks to the job farthest below its share as a ratio of its share. > This is easier to combine with other scheduling algorithms and leads to a > more stable sharing situation, avoiding unfairness issues brought up in > MAPREDUCE-543 and MAPREDUCE-544 that happen when some jobs have long tasks. > The new preemption (MAPREDUCE-551) will ensure that critical jobs can gain > their fair share within a bounded amount of time. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.