Hi all,
I have launched an application on yarn cluster which has following config.
Master (Resource Manager) - 16GB RAM + 8 vCPU
Slave 1 (Node manager 1) - 8GB RAM + 4 vCPU
Intermittently AM(2GB, 1 core) is exiting with code - 2 with the following
trace. I am not able to find anything about exit code 2.
Last log is
org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl:
Memory usage of ProcessTree 22504 for container-id
container_1469709900068_0002_01_000001: 203.8 MB of 2 GB physical memory used;
2.8 GB of 4.2 GB virtual memory used
Does this have anything to do with my application logic or Is it possible that
it is killed because of exceeding the memory limits?
2016-07-28 17:08:50,672 WARN
org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Exception
from container-launch with container ID: container_1469709900068_0002_01_000001
and exit code: 2
ExitCodeException exitCode=2:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
at org.apache.hadoop.util.Shell.run(Shell.java:455)
at
org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715)
at
org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:212)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-07-28 17:08:50,674 INFO
org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: Exception from
container-launch.
2016-07-28 17:08:50,674 INFO
org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: Container id:
container_1469709900068_0002_01_000001
2016-07-28 17:08:50,674 INFO
org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: Exit code: 2
2016-07-28 17:08:50,674 INFO
org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: Stack trace:
ExitCodeException exitCode=2:
Thanks,
Rahul Chhiber
