Re: yarn NoClassDefFoundError from LinuxContainerExecutor
Hi Mingjie, I don't know about Yarn, but NoClassDefFoundError appear when you have a class that was present during compile time but no longer available during runtime. See a detailed explanation here [1]. Check that the classpath built in the container/node contains the classes from that error. Also check that you don't get another version on the classpath, one without the specified class. Hope this helps, [1] http://javarevisited.blogspot.com/2011/06/noclassdeffounderror-exception-in.html Pe 29.02.2012 23:07, Mingjie Lai a scris: Hi. I'm trying yarn + security but still cannot make a mapred example runing. Can anyone help me to take a look? My env: - 3-slave cluster on ec2. Centos 5.5 - nn, dn, rm, nm all started, with security enabled. - i saw java.lang.NoClassDefFoundError from LinuxContainerExecutor eror log: ./application_1330545370212_0004/container_1330545370212_0004_01_01/stderr - If i disable security, still saw this issue. Any hint? I followed the instructions from http://hadoop.apache.org/common/docs/r0.23.0/hadoop-yarn/hadoop-yarn-site/ClusterSetup.html Steps: I started a mapred sample from nn/rm: $ usr/lib/hadoop/bin/yarn --config ./conf jar share/hadoop/mapreduce/hadoop-mapreduce-examples-0.24.0-SNAPSHOT.jar randomwriter 10 10 Logs are from nn, nm, -- [yarn@ip-10-176-231-35 hadoop]$ /usr/lib/hadoop/bin/yarn --config ./conf jar share/hadoop/mapreduce/hadoop-mapreduce-examples-0.24.0-SNAPSHOT.jar randomwriter 10 10 Running 30 maps. Job started: Wed Feb 29 20:33:48 UTC 2012 12/02/29 20:33:48 WARN conf.Configuration: mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used 12/02/29 20:33:49 INFO mapreduce.JobSubmitter: number of splits:30 12/02/29 20:33:49 INFO mapred.ResourceMgrDelegate: Submitted application application_1330545370212_0005 to ResourceManager at ip-10-176-231-35.us-west-1.compute.internal/10.176.231.35:7090 12/02/29 20:33:49 INFO mapreduce.Job: The url to track the job: http://ip-10-176-231-35.us-west-1.compute.internal:7050/proxy/application_1330545370212_0005/ 12/02/29 20:33:49 INFO mapreduce.Job: Running job: job_1330545370212_0005 12/02/29 20:33:53 INFO mapreduce.Job: Job job_1330545370212_0005 running in uber mode : false 12/02/29 20:33:53 INFO mapreduce.Job: map 0% reduce 0% 12/02/29 20:33:53 INFO mapreduce.Job: Job job_1330545370212_0005 failed with state FAILED due to: Application application_1330545370212_0005 failed 1 times due to AM Container for appattempt_1330545370212_0005_01 exited with exitCode: 1 due to: Exception from container-launch: org.apache.hadoop.util.Shell$ExitCodeException: at org.apache.hadoop.util.Shell.runCommand(Shell.java:261) at org.apache.hadoop.util.Shell.run(Shell.java:188) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381) at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:207) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:241) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:68) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) main : command provided 1 main : user is yarn .Failing this attempt.. Failing the application. 12/02/29 20:33:53 INFO mapreduce.Job: Counters: 0 Job ended: Wed Feb 29 20:33:53 UTC 2012 The job took 5 seconds. -- LinuxContainer error: [root@ip-10-176-203-45 yarn]# more ./application_1330545370212_0004/container_1330545370212_0004_01_01/stderr Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/yar n/service/CompositeService at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631) at java.lang.ClassLoader.defineClass(ClassLoader.java:615) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:14 1) at java.net.URLClassLoader.defineClass(URLClassLoader.java:283) at java.net.URLClassLoader.access$000(URLClassLoader.java:58) at java.net.URLClassLoader$1.run(URLClassLoader.java:197) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:306) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:247) Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.yarn.service.Comp ositeService at java.net.URLClassLoader$1.run(URLClassLoader.java:202) at java.security.AccessCo
Re: yarn NoClassDefFoundError from LinuxContainerExecutor
It relates to: https://issues.apache.org/jira/browse/MAPREDUCE-3505 Thanks. On 03/01/2012 07:09 AM, Ioan Eugen Stan wrote: Hi Mingjie, I don't know about Yarn, but NoClassDefFoundError appear when you have a class that was present during compile time but no longer available during runtime. See a detailed explanation here [1]. Check that the classpath built in the container/node contains the classes from that error. Also check that you don't get another version on the classpath, one without the specified class. Hope this helps, [1] http://javarevisited.blogspot.com/2011/06/noclassdeffounderror-exception-in.html Pe 29.02.2012 23:07, Mingjie Lai a scris: Hi. I'm trying yarn + security but still cannot make a mapred example runing. Can anyone help me to take a look? My env: - 3-slave cluster on ec2. Centos 5.5 - nn, dn, rm, nm all started, with security enabled. - i saw java.lang.NoClassDefFoundError from LinuxContainerExecutor eror log: ./application_1330545370212_0004/container_1330545370212_0004_01_01/stderr - If i disable security, still saw this issue. Any hint? I followed the instructions from http://hadoop.apache.org/common/docs/r0.23.0/hadoop-yarn/hadoop-yarn-site/ClusterSetup.html Steps: I started a mapred sample from nn/rm: $ usr/lib/hadoop/bin/yarn --config ./conf jar share/hadoop/mapreduce/hadoop-mapreduce-examples-0.24.0-SNAPSHOT.jar randomwriter 10 10 Logs are from nn, nm, -- [yarn@ip-10-176-231-35 hadoop]$ /usr/lib/hadoop/bin/yarn --config ./conf jar share/hadoop/mapreduce/hadoop-mapreduce-examples-0.24.0-SNAPSHOT.jar randomwriter 10 10 Running 30 maps. Job started: Wed Feb 29 20:33:48 UTC 2012 12/02/29 20:33:48 WARN conf.Configuration: mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used 12/02/29 20:33:49 INFO mapreduce.JobSubmitter: number of splits:30 12/02/29 20:33:49 INFO mapred.ResourceMgrDelegate: Submitted application application_1330545370212_0005 to ResourceManager at ip-10-176-231-35.us-west-1.compute.internal/10.176.231.35:7090 12/02/29 20:33:49 INFO mapreduce.Job: The url to track the job: http://ip-10-176-231-35.us-west-1.compute.internal:7050/proxy/application_1330545370212_0005/ 12/02/29 20:33:49 INFO mapreduce.Job: Running job: job_1330545370212_0005 12/02/29 20:33:53 INFO mapreduce.Job: Job job_1330545370212_0005 running in uber mode : false 12/02/29 20:33:53 INFO mapreduce.Job: map 0% reduce 0% 12/02/29 20:33:53 INFO mapreduce.Job: Job job_1330545370212_0005 failed with state FAILED due to: Application application_1330545370212_0005 failed 1 times due to AM Container for appattempt_1330545370212_0005_01 exited with exitCode: 1 due to: Exception from container-launch: org.apache.hadoop.util.Shell$ExitCodeException: at org.apache.hadoop.util.Shell.runCommand(Shell.java:261) at org.apache.hadoop.util.Shell.run(Shell.java:188) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381) at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:207) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:241) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:68) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) main : command provided 1 main : user is yarn .Failing this attempt.. Failing the application. 12/02/29 20:33:53 INFO mapreduce.Job: Counters: 0 Job ended: Wed Feb 29 20:33:53 UTC 2012 The job took 5 seconds. -- LinuxContainer error: [root@ip-10-176-203-45 yarn]# more ./application_1330545370212_0004/container_1330545370212_0004_01_01/stderr Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/yar n/service/CompositeService at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631) at java.lang.ClassLoader.defineClass(ClassLoader.java:615) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:14 1) at java.net.URLClassLoader.defineClass(URLClassLoader.java:283) at java.net.URLClassLoader.access$000(URLClassLoader.java:58) at java.net.URLClassLoader$1.run(URLClassLoader.java:197) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:306) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:247) Caused by: java.lang.ClassNotFoundException: org