[
https://issues.apache.org/jira/browse/MAPREDUCE-3451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13241368#comment-13241368
]
Ahmed Radwan commented on MAPREDUCE-3451:
-----------------------------------------
Thanks Patrick.
The patch successfully applies and compiles on trunk. The included new tests
also successfully run.
I'll postpone specific code comments, since I have some basic questions:
The old fair scheduler used slots as the basic allocation units, and
calculating shares and other features (like preemption) relied on this model.
In MR2 we have containers with variable resource attributes. Currently the only
supported attribute is memory, but it is planned to add more attributes
(network bandwidth, cpu, etc.). So it is not clear for me how this mapping is
handled. Are you envisioning combing such attributes for calculating shares,
how is this done? and how preemption will be done in such case, for example in
the old MR1, the most recent slot was killed. But now slots are not equal and
this complicate the choices. I am just using preemption as an example. Can you
please elaborate on that?
I also see that the patch doesn't include the web ui components. I think
postponing them to follow-up work is fine. But if someone wants to test the
patch on a real cluster, can you please highlight the steps needed to setup a
cluster using the new fair scheduler and if there are changes in setup compared
to the old MR1 version.
> Port Fair Scheduler to MR2
> --------------------------
>
> Key: MAPREDUCE-3451
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3451
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Components: mrv2, scheduler
> Reporter: Patrick Wendell
> Assignee: Patrick Wendell
> Attachments: MAPREDUCE-3451.v1.patch.txt, MAPREDUCE-3451.v2.patch.txt
>
>
> The Fair Scheduler is in widespread use today in MR1 clusters, but not yet
> ported to MR2. This is to track the porting of the Fair Scheduler to MR2 and
> will be updated to include design considerations and progress.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
