Hi Libo,
MRAppMaster is not able to load the yarn related jar files.
Is this the classpath using by MRAppMaster or any other process?
"/opt/hadoop/hadoop-2.0.0-cdh4.3.0/conf:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/*:/contrib/capacity-scheduler/*.jar:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce2/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce/*"
If this is not the MRAppMaster process classpath, can you try to get the
MRAppMaster java process classpath.
Thanks
Devaraj k
From: Yu, Libo [mailto:libo...@citi.com]
Sent: 11 July 2013 01:30
To: 'user@hadoop.apache.org'
Subject: NoClassDefFoundError: org/apache/hadoop/yarn/service/CompositeService
Hi,
I tried to run the wordcount example with yarn. Here is the command line:
hadoop jar share/hadoop/mapreduce2/hadoop-mapreduce-examples-2.0.0-cdh4.3.0.jar
wordcount /user/lyu/wordcount/input /user/lyu/wordcount/output
But I got this exception:
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/yar
n/service/CompositeService
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
at
java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
Caused by: java.lang.ClassNotFoundException:
org.apache.hadoop.yarn.service.CompositeService
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
... 12 more
Could not find the main class: org.apache.hadoop.mapreduce.v2.app.MRAppMaster.
Program will exit
Here is conent of the CLASSPATH used by java:
/opt/hadoop/hadoop-2.0.0-cdh4.3.0/conf:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/*:/contrib/capacity-scheduler/*.jar:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce2/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce/*
Here is the console output:
13/07/09 20:43:07 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
13/07/09 20:43:07 INFO service.AbstractService:
Service:org.apache.hadoop.yarn.client.YarnClientImpl is inited.
13/07/09 20:43:07 INFO service.AbstractService:
Service:org.apache.hadoop.yarn.client.YarnClientImpl is started.
13/07/09 20:43:08 INFO input.FileInputFormat: Total input paths to process : 0
13/07/09 20:43:08 INFO mapreduce.JobSubmitter: number of splits:0
13/07/09 20:43:08 WARN conf.Configuration: mapred.jar is deprecated. Instead,
use mapreduce.job.jar
13/07/09 20:43:08 WARN conf.Configuration: mapred.output.value.class is
deprecated. Instead, use mapreduce.job.output.value.class
13/07/09 20:43:08 WARN conf.Configuration: mapreduce.combine.class is
deprecated. Instead, use mapreduce.job.combine.class
13/07/09 20:43:08 WARN conf.Configuration: mapreduce.map.class is deprecated.
Instead, use mapreduce.job.map.class
13/07/09 20:43:08 WARN conf.Configuration: mapred.job.name is deprecated.
Instead, use mapreduce.job.name
13/07/09 20:43:08 WARN conf.Configuration: mapreduce.reduce.class is
deprecated. Instead, use mapreduce.job.reduce.class
13/07/09 20:43:08 WARN conf.Configuration: mapred.input.dir is deprecated.
Instead, use mapreduce.input.fileinputformat.inputdir
13/07/09 20:43:08 WARN conf.Configuration: mapred.output.dir is deprecated.
Instead, use mapreduce.output.fileoutputformat.outputdir
13/07/09 20:43:08 WARN conf.Configuration: mapred.map.tasks is deprecated.
Instead, use mapreduce.job.maps
13/07/09 20:43:08 WARN conf.Configuration: mapred.output.key.class is
deprecated. Instead, use mapreduce.job.output.key.class
13/07/09 20:43:08 WARN conf.Configuration: mapred.working.dir is deprecated.
Instead, use mapreduce.job.working.dir
13/07/09 20:43:09 INFO mapreduce.JobSubmitter: Submitting tokens for job:
job_1373416798858_0001
13/07/09 20:43:09 INFO client.YarnClientImpl: Submitted application
application_1373416798858_0001 to ResourceManager at
host1.libo.com/192.168.10.1:8032
13/07/09 20:43:09 INFO mapreduce.Job: The url to track the job:
http://host1.libo.com:8088/proxy/application_1373416798858_0001/
13/07/09 20:43:09 INFO mapreduce.Job: Running job: job_1373416798858_0001
13/07/09 20:43:12 INFO mapreduce.Job: Job job_1373416798858_0001 running in
uber mode : false
13/07/09 20:43:12 INFO mapreduce.Job: map 0% reduce 0%
13/07/09 20:43:12 INFO mapreduce.Job: Job job_1373416798858_0001 failed with
state FAILED due to: Application application_1373416798858_0001 failed 1 times
due to AM Container for appattempt_1373416798858_0001_000001 exited with
exitCode: 1 due to:
.Failing this attempt.. Failing the application.
13/07/09 20:43:12 INFO mapreduce.Job: Counters: 0
The paths for CompositeService and MRAppMaster are in the CLASSPATH and I
checked the jar files and did
find the class files. Anybody knows why? Thanks.
Regards,
Libo
