Hi, all: This is the log of my nodemanager, I set 4GB memory for memory task and virtual memory ratio to 2.5, so the total virtual memory is 10GB.
*log*: org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 725 for container-id container_1367846809555_0003_01_000002: 943.9 MB of 4 GB physical memory used; 2.4 GB of 10 GB virtual memory used 1)Do you know how yarn calculate the actual physical and virtual memory usage? How can I add log to the container and print the usage for each container? 2)BTW, I set task input to 1024mb and io.sort.mb to 512mb for experiment, however, I find the total actual physical memory waste for about 900mb. Why it looks like this, doesn't it will spill when the memory consumption is more than 512(io.sort.mb)? 3)How can I set memory and cpu for hadoop sort example to make full use of resource? I have 24GB and 24core per machine, the input size is 256mb, so can I set the vcores to less than *1*? -- *Sincerely,* *Zhaojie* * *
