[
https://issues.apache.org/jira/browse/MAPREDUCE-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12935001#action_12935001
]
Scott Chen commented on MAPREDUCE-2198:
---------------------------------------
Hey Arun and Joydeep,
bq. How do you handle heterogeneous clusters? Or will your CLI command be per
TT?
Yes, the CLI command will be per TT.
bq. in addition - one bug/feature that we need to fix as part of this where the
JT overschedules TTs (when configured to schedule multiple tasks per hbt). this
is benign today (TT puts those tasks in unassigned state) - but in this world
will not be so benign.
Yes, we should never let JT submit more tasks than TT's limit. Scheduler should
incorporate this logic.
bq. You also have to worry about piggy-backing of task-cleanup tasks done by
the JT...
I am not very clear about this part. Will read more codes.
> Allow FairScheduler to control the number of slots on each TaskTracker
> ----------------------------------------------------------------------
>
> Key: MAPREDUCE-2198
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2198
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Components: contrib/fair-share
> Affects Versions: 0.22.0
> Reporter: Scott Chen
> Assignee: Scott Chen
> Fix For: 0.22.0
>
>
> We can set the number of slots on the TaskTracker to be high and let
> FairScheduler handles the slots.
> This approach allows us to change the number of slots on each node
> dynamically.
> The administrator can change the number of slots with a CLI tool.
> One use case of this is for upgrading the MapReduce.
> Instead of restarting the cluster, we can run the new MapReduce on the same
> cluster.
> And use the CLI tool to gradually migrate the slots.
> This way we don't lost the progress fo the jobs that's already executed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
