[
https://issues.apache.org/jira/browse/MAPREDUCE-2591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13049186#comment-13049186
]
Amar Kamat commented on MAPREDUCE-2591:
---------------------------------------
Adding details:
1. JVM reuse: Here the total cpu usage of the process tree will be a net sum of
all the other tasks that ran in the current JVM. This information is not known
to the task. Cpu usage emulator should identify such cases and emulate on top
of the current cpu usage.
2. Short lived maps emulating high CPU usage: If the total span of the map task
is short, then the task will exit before emulating the resource usage. The
framework (or the emulator) should detect such situations (based on map input
data size, input records, features like compression etc) and emulate
aggressively. On way to do this would be to give more weight in the early
phases of the task.
3. Long running maps/reducers with high CPU usage: If the total runtime of the
task is high and the total expected cpu usage is high, then the emulator should
speed up its emulation phase. One option can be to use multi-threading. Other
option would be to switch to a more cpu intensive operation(s).
> [Gridmix] Improve cumulative CPU usage emulation
> ------------------------------------------------
>
> Key: MAPREDUCE-2591
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2591
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: contrib/gridmix
> Reporter: Amar Kamat
> Assignee: Amar Kamat
> Fix For: 0.23.0
>
>
> MAPREDUCE-2106 introduced a basic cumulative cpu usage emulation in Gridmix.
> There are some known cases where it can be improved. The cases are
> 1. JVM reuse
> 2. Short lived maps emulating high CPU usage
> 3. Long running maps/reducers with high CPU usage
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
