Matei, Are you sure you are using the latest common from trunk? The ClientCache was recently added within the last few weeks. Looks like its using some old version of hadoop common.
Also the sl4j errors should be fixed in the latest MR-279 branch. thanks mahadev On Aug 11, 2011, at 1:52 PM, Matei Zaharia wrote: > I get a similar error on Mac OS X. I've built YARN and extracted the tarball > to a directory, but when I run bin/yarn-daemon.sh start nodemanager, it > prints the following to its log (apologies for the long trace): > > log4j:WARN No appenders could be found for logger > (org.apache.hadoop.metrics2.impl.MetricsSystemImpl). > log4j:WARN Please initialize the log4j system properly. > log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more > info. > Exception in thread "main" org.apache.hadoop.yarn.YarnException: Failed to > Start org.apache.hadoop.yarn.server.nodemanager.NodeManager > at > org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:80) > at > org.apache.hadoop.yarn.server.nodemanager.NodeManager.start(NodeManager.java:146) > at > org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:191) > Caused by: org.apache.avro.AvroRuntimeException: > org.apache.hadoop.yarn.YarnException: > java.lang.reflect.InvocationTargetException > at > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.start(NodeStatusUpdaterImpl.java:140) > at > org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:68) > ... 2 more > Caused by: org.apache.hadoop.yarn.YarnException: > java.lang.reflect.InvocationTargetException > at > org.apache.hadoop.yarn.factories.impl.pb.RpcClientFactoryPBImpl.getClient(RpcClientFactoryPBImpl.java:70) > at > org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC.getProxy(HadoopYarnProtoRPC.java:35) > at > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.getRMClient(NodeStatusUpdaterImpl.java:158) > at > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.registerWithRM(NodeStatusUpdaterImpl.java:163) > at > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.start(NodeStatusUpdaterImpl.java:136) > ... 3 more > Caused by: java.lang.reflect.InvocationTargetException > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) > at java.lang.reflect.Constructor.newInstance(Constructor.java:513) > at > org.apache.hadoop.yarn.factories.impl.pb.RpcClientFactoryPBImpl.getClient(RpcClientFactoryPBImpl.java:67) > ... 7 more > Caused by: java.lang.NoClassDefFoundError: org/apache/hadoop/ipc/ClientCache > at > org.apache.hadoop.yarn.ipc.ProtoOverHadoopRpcEngine.<clinit>(ProtoOverHadoopRpcEngine.java:63) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:247) > at > org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1108) > at > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1160) > at org.apache.hadoop.ipc.RPC.getProtocolEngine(RPC.java:94) > at org.apache.hadoop.ipc.RPC.getProtocolProxy(RPC.java:422) > at org.apache.hadoop.ipc.RPC.getProtocolProxy(RPC.java:368) > at org.apache.hadoop.ipc.RPC.getProtocolProxy(RPC.java:333) > at org.apache.hadoop.ipc.RPC.getProtocolProxy(RPC.java:461) > at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:442) > at > org.apache.hadoop.yarn.server.api.impl.pb.client.ResourceTrackerPBClientImpl.<init>(ResourceTrackerPBClientImpl.java:32) > ... 12 more > Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.ipc.ClientCache > at java.net.URLClassLoader$1.run(URLClassLoader.java:202) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:190) > at java.lang.ClassLoader.loadClass(ClassLoader.java:306) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) > at java.lang.ClassLoader.loadClass(ClassLoader.java:247) > ... 24 more > > In addition, if I try to start resourcemanager instead, it logs: > > SLF4J: Class path contains multiple SLF4J bindings. > SLF4J: Found binding in > [jar:file:/Users/matei/workspace/MR-279/common/build/ivy/lib/Hadoop-Common/common/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] > SLF4J: Found binding in > [jar:file:/Users/matei/workspace/MR-279-deploy/hadoop-mapreduce-1.0-SNAPSHOT/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an > explanation. > log4j:WARN No appenders could be found for logger > (org.apache.hadoop.metrics2.lib.MutableMetricsFactory). > log4j:WARN Please initialize the log4j system properly. > log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more > info. > Exception in thread "main" java.lang.IllegalStateException: For this > operation, current State must be STARTED instead of INITED > at > org.apache.hadoop.yarn.service.AbstractService.ensureCurrentState(AbstractService.java:101) > at > org.apache.hadoop.yarn.service.AbstractService.stop(AbstractService.java:69) > at > org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher.stop(ApplicationMasterLauncher.java:90) > at > org.apache.hadoop.yarn.service.CompositeService.stop(CompositeService.java:89) > at > org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.stop(ResourceManager.java:423) > at > org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:493) > > I don't know where the multiple SLF4J versions are coming from because I > checked out common, hdfs and mapreduce at the same time. > > Matei > > > > > On Aug 11, 2011, at 2:01 AM, Vinod KV wrote: > >> >> >> Report YARN_* and HADOOP_* variables that you've set before starting the >> daemons. >> >> Also run 'bin/yarn classpath' and post the output of that command too. If >> the output doesn't contain yarn-server-resourcemanager-1.0-SNAPSHOT.jar, you >> are missing something. >> >> +Vinod >> >> >> On Thursday 11 August 2011 01:57 AM, Haoyuan Li wrote: >>> Hi, >>> >>> When I ran ResourceManager or NodeManager as steps here >>> http://svn.apache.org/repos/asf/hadoop/common/branches/MR-279/mapreduce/INSTALL >>> . >>> It has main class can't be found exception... I attached the shell output >>> here. Any help will be appreciated! >>> >>> Thanks, >>> >>> Haoyuan >>> >>> haoyuan@hya:~/hadoop/hadoop-mapreduce-1.0-SNAPSHOT$ ./bin/yarn-daemon.sh >>> start resourcemanager >>> starting resourcemanager, logging to >>> /home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../logs/yarn-haoyuan-resourcemanager-hya.out >>> /usr/lib/jvm/java-6-sun/bin/java -Dproc_resourcemanager -Xmx1000m >>> -Dhadoop.log.dir=/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../logs >>> -Dyarn.log.dir=/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../logs >>> -Dhadoop.log.file=yarn-haoyuan-resourcemanager-hya.log >>> -Dyarn.log.file=yarn-haoyuan-resourcemanager-hya.log -Dyarn.home.dir= >>> -Dyarn.id.str=haoyuan -Dhadoop.root.logger=INFO,DRFA >>> -Dyarn.root.logger=INFO,DRFA -Dyarn.policy.file=hadoop-policy.xml >>> -Dhadoop.log.dir=/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../logs >>> -Dyarn.log.dir=/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../logs >>> -Dhadoop.log.file=yarn-haoyuan-resourcemanager-hya.log >>> -Dyarn.log.file=yarn-haoyuan-resourcemanager-hya.log >>> -Dyarn.home.dir=/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/.. >>> -Dhadoop.root.logger=INFO,DRFA -Dyarn.root.logger=INFO,DRFA -classpath >>> /home/haoyuan/hadoop/conf:/home/haoyuan/hadoop/conf:/usr/lib/jvm/java-6-sun/lib/tools.jar:/home/haoyuan/hadoop/trunk/common/hadoop-*.jar:/home/haoyuan/hadoop/trunk/common/lib/*.jar:/home/haoyuan/hadoop/trunk/common/share/hadoop/common/*.jar:/home/haoyuan/hadoop/trunk/common/share/hadoop/common/lib/*.jar:/home/haoyuan/hadoop/trunk/common/share/hadoop/hdfs/*.jar:/home/haoyuan/hadoop/trunk/hdfs/hadoop-*.jar:/home/haoyuan/hadoop/trunk/hdfs/lib/*.jar:/home/haoyuan/hadoop/trunk/hdfs/build/classes:/home/haoyuan/hadoop/trunk/hdfs/hadoop-*.jar:/home/haoyuan/hadoop/trunk/hdfs/lib/*.jar:/home/haoyuan/hadoop/trunk/mapreduce/build/classes:/home/haoyuan/hadoop/trunk/mapreduce/build:/home/haoyuan/hadoop/trunk/mapreduce/build/test/classes:/home/haoyuan/hadoop/trunk/mapreduce/build/tools:/home/haoyuan/hadoop/trunk/mapreduce/lib/*.jar:/home/haoyuan/hadoop/trunk/mapreduce/*.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/aopalliance-1.0.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/asm-3.2.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/aspectjrt-1.6.5.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/avro-1.3.2.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/avro-1.4.1.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/clover-3.0.2.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/commons-beanutils-1.7.0.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/commons-beanutils-core-1.8.0.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/commons-cli-1.2.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/commons-codec-1.4.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/commons-collections-3.2.1.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/commons-configuration-1.6.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/commons-digester-1.8.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/commons-httpclient-3.1.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/commons-lang-2.5.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/commons-logging-1.0.4.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/commons-logging-api-1.0.4.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/commons-math-2.1.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/commons-net-1.4.1.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/core-3.1.1.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/guava-r09.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/guice-2.0.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/guice-servlet-2.0.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/hadoop-annotations-0.23.0-SNAPSHOT.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/jackson-core-asl-1.4.2.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/jackson-mapper-asl-1.4.2.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/jdiff-1.0.9.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/jets3t-0.6.1.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/jetty-6.1.26.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/jetty-util-6.1.26.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/jsp-api-2.1.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/junit-4.8.2.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/kfs-0.3.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/log4j-1.2.15.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/netty-3.2.3.Final.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/oro-2.0.8.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/paranamer-2.2.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/paranamer-ant-2.2.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/paranamer-generator-2.2.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/protobuf-java-2.4.0a.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/qdox-1.10.1.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/servlet-api-2.5.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/slf4j-api-1.6.1.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/slf4j-log4j12-1.6.1.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/xmlenc-0.52.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../modules/hadoop-mapreduce-client-app-1.0-SNAPSHOT.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../modules/hadoop-mapreduce-client-common-1.0-SNAPSHOT.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../modules/hadoop-mapreduce-client-core-1.0-SNAPSHOT.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../modules/hadoop-mapreduce-client-hs-1.0-SNAPSHOT.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../modules/hadoop-mapreduce-client-jobclient-1.0-SNAPSHOT.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../modules/hadoop-mapreduce-client-shuffle-1.0-SNAPSHOT.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../modules/yarn-api-1.0-SNAPSHOT.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../modules/yarn-common-1.0-SNAPSHOT.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../modules/yarn-server-common-1.0-SNAPSHOT.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../modules/yarn-server-nodemanager-1.0-SNAPSHOT.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../modules/yarn-server-resourcemanager-1.0-SNAPSHOT.jar:/home/haoyuan/hadoop/conf/rm-config/log4j.properties >>> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager >>> Exception in thread "main" java.lang.NoClassDefFoundError: >>> org/apache/hadoop/conf/Configuration >>> Caused by: java.lang.ClassNotFoundException: >>> org.apache.hadoop.conf.Configuration >>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202) >>> at java.security.AccessController.doPrivileged(Native Method) >>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:306) >>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:247) >>> Could not find the main class: >>> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager. Program will >>> exit. >>> >> >> >
