[
https://issues.apache.org/jira/browse/MAPREDUCE-961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12753136#action_12753136
]
dhruba borthakur commented on MAPREDUCE-961:
--------------------------------------------
> 1. Are you planning to use an existing monitoring system or create your own?
Most exiting monitoring systems cannot be used as it is because they do not
report statistics based on hadoop jobs. The system has to report CPU/memory etc
per hadoop job.
> 2. Will this make mapred.tasktracker.{map|reduce}.tasks.maximum obsolete?
This is very likely to happen. I imagine that the scheduler can continue to
schedule tasks (even though it exceeds may
mapred.tasktracker.{map|reduce}.tasks.maximum because it can find that there is
plenty of CPU left on a particular slave machine).
> ResourceAwareLoadManager to dynamically decide new tasks based on current
> CPU/memory load on TaskTracker(s)
> -----------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-961
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-961
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Components: contrib/fair-share
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
>
> Design and develop a ResouceAwareLoadManager for the FairShare scheduler that
> dynamically decides how many maps/reduces to run on a particular machine
> based on the CPU/Memory/diskIO/network usage in that machine. The amount of
> resources currently used on each task tracker is being fed into the
> ResourceAwareLoadManager in real-time via an entity that is external to
> Hadoop.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
