Re: How are 'PHYSICAL_MEMORY_BYTES' and 'VIRTUAL_MEMORY_BYTES' calculated?

2013-07-12 Thread Vinod Kumar Vavilapalli
They are the running metrics. While the task is running, they will tell you how much pmem/vmem it is using at that point of time. Obviously at the end of job, it will be the last snapshot. Thanks, +Vinod On Jul 12, 2013, at 6:47 AM, Shahab Yunus wrote: I think they are cumulative but per

Re: How are 'PHYSICAL_MEMORY_BYTES' and 'VIRTUAL_MEMORY_BYTES' calculated?

2013-07-12 Thread hadoop qi
Thanks for the response. So they represent the total physical memory (virtual memory) has been allocated to the job (e.g., from heap and stack) during its entire life time? I am still confused how to get the cumulative number from /proc/meminfo. I think from /proc/meminfo we can only get the

Re: How are 'PHYSICAL_MEMORY_BYTES' and 'VIRTUAL_MEMORY_BYTES' calculated?

2013-07-12 Thread Shahab Yunus
As Vinod Kumar Vavilapalli they are indeed snapshots in point and time. So they are neither the peak usage from the whole duration of the job, nor cumulative aggregate that increases over time. Regards, Shahab On Fri, Jul 12, 2013 at 4:47 PM, hadoop qi hadoop@gmail.com wrote: Thanks for

Re: How are 'PHYSICAL_MEMORY_BYTES' and 'VIRTUAL_MEMORY_BYTES' calculated?

2013-07-12 Thread Vinod Kumar Vavilapalli
No, every so often, 3 seconds IIRC, it capture pmem and vmem which corresponds to the usage of the process and its children at *that* specific point of time. Cumulative = cumulative across the process and its children. Thanks, +Vinod On Jul 12, 2013, at 1:47 PM, hadoop qi wrote: Thanks for

How are 'PHYSICAL_MEMORY_BYTES' and 'VIRTUAL_MEMORY_BYTES' calculated?

2013-07-11 Thread hadoop qi
Hello, I am wondering how memory counters 'PHYSICAL_MEMORY_BYTES' and 'VIRTUAL_MEMORY_BYTES' are calculated? They are peaks of memory usage or cumulative usage? Thanks for help,