[
https://issues.apache.org/jira/browse/MAPREDUCE-2469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13034681#comment-13034681
]
Amar Kamat commented on MAPREDUCE-2469:
---------------------------------------
The patch looks fine to me. For the test case, can we do the following
1. Start a Mini-MR cluster.
2. Submit a no-op job (say job1) with 1 map and 1 reduce
3. Submit a word count like job (say job2) with 1 map and 1 reduce.
4. Check if job1's heap usage (reported via its counters) is less than that of
job2.
For (3), any job that can add some extra load in the task's heap should
suffice. Also both the jobs should have 'mapred.child.java.opts' having -Xms
set to a very low value.
Thoughts?
> Task counters should also report the total heap usage of the task
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-2469
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2469
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: task
> Affects Versions: 0.23.0
> Reporter: Amar Kamat
> Assignee: Ravi Gummadi
> Labels: mapreduce
> Attachments: 2469.v0.1.patch
>
>
> Currently, the task counters report VSS and RSS usage of the task. The task
> counter should also report the total heap usage of the task also. The task
> might be configured with a max heap size of M but the task's total heap usage
> might only be H, where H < M. In such a case, knowing only M doesn't provide
> a complete picture of the task's memory usage.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
