Arun, thank you for the tips.
I viewed LinuxResourceCalculatorPlugin. I think it calculates the
resource usage of a node, since it parses the log files under /proc/.
But on a single node there can be multiple containers running.
Can you point out in which classes the resource usage of a single
container is monitored or controlled? Thank you very much!
On 6/21/2013 4:50 PM, Arun C Murthy wrote:
Start with LinuxResourceCalculatorPlugin.
Arun
On Jun 21, 2013, at 1:30 PM, Yuzhang Han <yuzhanghan1...@gmail.com> wrote:
Thank you Arun.
I am trying to study how YARN works. Can someone tell me which class(es)
monitors containers w.r.t. memory usage?
On 6/18/2013 4:07 PM, Arun C Murthy wrote:
NodeManagers monitor containers w.r.t memory usage, and put containers in
cgroups with cpu limits to restrict CPU usage.
On Jun 18, 2013, at 12:30 PM, Yuzhang Han <yuzhanghan1...@gmail.com> wrote:
Hi,
I am curious about how YARN containers control their memory usage. Say, I have
a MR job, and I configure that every map task should be assigned a 1 GB
container, and every reduce task a 1.5 GB one. So, when YARN runs the
containers, how is it ensured that all map containers use less than 1 GB memory
and all reduce containers less than 1.5 GB? Is container memory control
realized via -Xmx arguments in the scripts generated by YARN to invoke the MR
program in the containers?
Thank you.
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
