[
https://issues.apache.org/jira/browse/MAPREDUCE-2653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13063731#comment-13063731
]
nandan commented on MAPREDUCE-2653:
-----------------------------------
To develop a proof of concept I have just concentrated on a cpu utilization.
Currently, I am running multiple jobs simultaneously.
Based on cpu utilization of the task and current cpu idle time, I decide if I
can run an extra task of that job (by dynamically increasing map slots),
thereby coupling cpu intensive jobs along with jobs which are not cpu intensive
> dynamic map slots (in addition to predifined) on each node which allows to
> execute cpu intensive jobs along with memory intensive jobs thereby reducing
> wastage of cpu cycles
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-2653
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2653
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker, tasktracker
> Affects Versions: 0.20.203.0
> Environment: linux
> Reporter: nandan
> Labels: map, scheduler, tasks
> Original Estimate: 672h
> Remaining Estimate: 672h
>
> I have introduced process monitoring system inside tasktracker, which
> analyses the cpu and memory utilization of each map task and allows me to
> increase/decrease maximum number of map slots dynamically on each node. With
> this I can combine cpu intensive jobs along with memory intensive jobs,
> thereby reducing the cpu idle time.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
