[ https://issues.apache.org/jira/browse/MAPREDUCE-4710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13864833#comment-13864833 ]
Ming Ma commented on MAPREDUCE-4710: ------------------------------------ A general question, should NM provide such data at the container level? It seems we need that information to support preemption and fairness anyway; NM needs to inform RM the actual resource utilization at container level; memory usage is one of the resource metrics. Currently ContainerStatus doesn't provide that level of details. > Add peak memory usage counter for each task > ------------------------------------------- > > Key: MAPREDUCE-4710 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4710 > Project: Hadoop Map/Reduce > Issue Type: New Feature > Components: task > Affects Versions: 1.0.2, trunk > Reporter: Cindy Li > Assignee: Cindy Li > Priority: Minor > Labels: patch > Fix For: trunk > > Attachments: MAPREDUCE-4710-trunk.patch, mapreduce-4710-v1.0.2.patch, > mapreduce-4710.patch, mapreduce4710-v3.patch, mapreduce4710-v6.patch, > mapreduce4710.patch > > > Each task has counters PHYSICAL_MEMORY_BYTES and VIRTUAL_MEMORY_BYTES, which > are snapshots of memory usage of that task. They are not sufficient for users > to understand peak memory usage by that task, e.g. in order to diagnose task > failures, tune job parameters or change application design. This new feature > will add two more counters for each task: PHYSICAL_MEMORY_BYTES_MAX and > VIRTUAL_MEMORY_BYTES_MAX. -- This message was sent by Atlassian JIRA (v6.1.5#6160)