Hi Mahadev, This works for me. Thank you.
However, another issue came to me when I tried to run an example. Here http://svn.apache.org/repos/asf/hadoop/common/branches/MR-279/mapreduce/INSTALL , it says: $HADOOP_COMMON_HOME/bin/hadoop jar $HADOOP_MAPRED_HOME/build/hadoop-mapred-examples-0.22.0-SNAPSHOT.jar randomwriter -Dmapreduce.job.user.name=$USER -Dmapreduce.clientfactory.class.name=org.apache.hadoop.mapred.YarnClientFactory -Dmapreduce.randomwriter.bytespermap=10000 -Ddfs.blocksize=536870912 -Ddfs.block.size=536870912 -libjars $HADOOP_YARN_INSTALL/hadoop-mapreduce-1.0-SNAPSHOT/modules/hadoop-mapreduce-client-jobclient-1.0-SNAPSHOT.jar output However, there is no /bin folder in $HADOOP_COMMON_HOME. I found /bin/hadoop in $HADOOP_COMMON_HOME/src/main/bin/hadoop. When I executed the command: ./hadoop-common/src/main/bin/hadoop jar /home/haoyuan/hadoop/trunk/mapreduce/build/hadoop-mapred-examples-0.23.0-SNAPSHOT.jar randomwriter -Dmapreduce.job.user.name=haoyuan - Dmapreduce.clientfactory.class.name=org.apache.hadoop.mapred.YarnClientFactory -Dmapreduce.randomwriter.bytespermap=10000 -Ddfs.blocksize=536870912 -Ddfs.block.size=536870912 -libjars /home/haoyuan/hadoop/trunk/hadoop-mapreduce-1.0-SNAPSHOT/modules/hadoop-mapreduce-client-jobclient-1.0-SNAPSHOT.jar output I got another exception: =========================================== haoyuan@hya:~/hadoop/trunk$ ./hadoop-common/src/main/bin/hadoop jar /home/haoyuan/hadoop/trunk/mapreduce/build/hadoop-mapred-examples-0.23.0-SNAPSHOT.jar randomwriter -Dmapreduce.job.user.name=haoyuan - Dmapreduce.clientfactory.class.name=org.apache.hadoop.mapred.YarnClientFactory -Dmapreduce.randomwriter.bytespermap=10000 -Ddfs.blocksize=536870912 -Ddfs.block.size=536870912 -libjars /home/haoyuan/hadoop/trunk/hadoop-mapreduce-1.0-SNAPSHOT/modules/hadoop-mapreduce-client-jobclient-1.0-SNAPSHOT.jar output Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/util/RunJar Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.util.RunJar at java.net.URLClassLoader$1.run(URLClassLoader.java:202) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:306) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:247) Could not find the main class: org.apache.hadoop.util.RunJar. Program will exit. =========================================== Any clue? Thank you. Best, Haoyuan On Thu, Aug 11, 2011 at 3:47 PM, Mahadev Konar <maha...@hortonworks.com>wrote: > Haoyuan, > Thats an issue with having avro 1.3.2 jar in the classpath. Please > see if you have that version of 1.3.2 jar in the classpath and remove > it. We use 1.4 avro jar which isnt compatible with 1.3.2. > > hope that helps! > mahadev > > On Thu, Aug 11, 2011 at 3:36 PM, Haoyuan Li <haoyuan...@gmail.com> wrote: > > I downloaded the latest version: r1156719. and then redid from the > scratch. > > Now, the nodemanager and run well. However, I got into the same state as > > Matei when I tried to run ResourceManager. Any help will be appreciated. > > Best, > > Haoyuan > > The followings are output. > > ============ The output from .out log file ================= > > Exception in thread "main" java.lang.IllegalStateException: For this > > operation, current State must be STARTED instead of INITED > > at > > > org.apache.hadoop.yarn.service.AbstractService.ensureCurrentState(AbstractService.java:101) > > at > > > org.apache.hadoop.yarn.service.AbstractService.stop(AbstractService.java:69) > > at > > > org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher.stop(ApplicationMasterLauncher.java:90) > > at > > > org.apache.hadoop.yarn.service.CompositeService.stop(CompositeService.java:89) > > at > > > org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.stop(ResourceManager.java:423) > > at > > > org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:495) > > ===================================================== > > =========== the output from .log log file =================== > > 2011-08-11 15:25:38,357 INFO > > org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Resource > > Manager is starting... > > 2011-08-11 15:25:38,878 INFO > org.apache.hadoop.yarn.event.AsyncDispatcher: > > Registering class > > > org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.SchedulerEventType > > for class > > > org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher > > 2011-08-11 15:25:38,879 INFO > org.apache.hadoop.yarn.event.AsyncDispatcher: > > Registering class > > org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEventType for > class > > > org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationEventDispatcher > > 2011-08-11 15:25:38,880 INFO > org.apache.hadoop.yarn.event.AsyncDispatcher: > > Registering class > > > org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEventType > > for class > > > org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationAttemptEventDispatcher > > 2011-08-11 15:25:38,881 INFO > org.apache.hadoop.yarn.event.AsyncDispatcher: > > Registering class > > org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeEventType for > > class > > > org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$NodeEventDispatcher > > 2011-08-11 15:25:38,904 INFO > org.apache.hadoop.yarn.event.AsyncDispatcher: > > Registering class > > > org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncherEventType > > for class > > > org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher > > 2011-08-11 15:25:38,907 INFO > org.apache.hadoop.yarn.service.AbstractService: > > Service:Dispatcher is inited. > > 2011-08-11 15:25:38,907 INFO > org.apache.hadoop.yarn.service.AbstractService: > > > Service:org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.ContainerAllocationExpirer > > is inited. > > 2011-08-11 15:25:38,935 INFO > org.apache.hadoop.yarn.service.AbstractService: > > Service:AMLivelinessMonitor is inited. > > 2011-08-11 15:25:38,936 INFO org.apache.hadoop.util.HostsFileReader: > > Refreshing hosts (include/exclude) list > > 2011-08-11 15:25:38,936 INFO > org.apache.hadoop.yarn.service.AbstractService: > > Service:org.apache.hadoop.yarn.server.resourcemanager.NodesListManager is > > inited. > > 2011-08-11 15:25:38,936 INFO > org.apache.hadoop.yarn.service.AbstractService: > > > Service:org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher > > is inited. > > 2011-08-11 15:25:38,936 INFO > org.apache.hadoop.yarn.service.AbstractService: > > Service:NMLivelinessMonitor is inited. > > 2011-08-11 15:25:38,940 INFO > org.apache.hadoop.yarn.service.AbstractService: > > > Service:org.apache.hadoop.yarn.server.resourcemanager.ResourceTrackerService > > is inited. > > 2011-08-11 15:25:38,941 INFO > org.apache.hadoop.yarn.service.AbstractService: > > > Service:org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService > > is inited. > > 2011-08-11 15:25:38,941 INFO > org.apache.hadoop.yarn.service.AbstractService: > > Service:org.apache.hadoop.yarn.server.resourcemanager.ClientRMService is > > inited. > > 2011-08-11 15:25:38,941 INFO > org.apache.hadoop.yarn.service.AbstractService: > > Service:org.apache.hadoop.yarn.server.resourcemanager.AdminService is > > inited. > > 2011-08-11 15:25:38,965 INFO > org.apache.hadoop.yarn.service.AbstractService: > > > Service:org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher > > is inited. > > 2011-08-11 15:25:38,965 INFO > org.apache.hadoop.yarn.service.AbstractService: > > Service:ResourceManager is inited. > > 2011-08-11 15:25:39,057 INFO org.mortbay.log: Logging to > > org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via > > org.mortbay.log.Slf4jLog > > 2011-08-11 15:25:39,125 INFO org.apache.hadoop.http.HttpServer: Added > global > > filter 'safety' > (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter) > > 2011-08-11 15:25:39,128 INFO org.apache.hadoop.http.HttpServer: Added > filter > > static_user_filter > > (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) > to > > context WepAppsContext > > 2011-08-11 15:25:39,128 INFO org.apache.hadoop.http.HttpServer: Added > filter > > static_user_filter > > (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) > to > > context static > > 2011-08-11 15:25:39,128 INFO org.apache.hadoop.http.HttpServer: Added > filter > > static_user_filter > > (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) > to > > context logs > > 2011-08-11 15:25:39,131 INFO org.apache.hadoop.http.HttpServer: Added > global > > filter 'guice' (class=com.google.inject.servlet.GuiceFilter) > > 2011-08-11 15:25:39,131 INFO org.apache.hadoop.http.HttpServer: Port > > returned by webServer.getConnectors()[0].getLocalPort() before open() is > -1. > > Opening the listener on 8088 > > 2011-08-11 15:25:39,136 INFO org.apache.hadoop.http.HttpServer: > > listener.getLocalPort() returned 8088 > > webServer.getConnectors()[0].getLocalPort() returned 8088 > > 2011-08-11 15:25:39,136 INFO org.apache.hadoop.http.HttpServer: Jetty > bound > > to port 8088 > > 2011-08-11 15:25:39,136 INFO org.mortbay.log: jetty-6.1.26 > > 2011-08-11 15:25:39,160 INFO org.mortbay.log: Extract > > > jar:file:/home/haoyuan/hadoop/trunk/hadoop-mapreduce-1.0-SNAPSHOT/modules/yarn-common-1.0-SNAPSHOT.jar!/webapps/yarn > > to /tmp/Jetty_0_0_0_0_8088_yarn____yzuv81/webapp > > 2011-08-11 15:25:39,238 INFO org.mortbay.log: NO JSP Support for /, did > not > > find org.apache.jasper.servlet.JspServlet > > 2011-08-11 15:25:39,319 INFO org.mortbay.log: Started > > SelectChannelConnector@0.0.0.0:8088 > > 2011-08-11 15:25:39,320 INFO org.apache.hadoop.yarn.webapp.WebApps: Web > app > > /yarn started at 8088 > > 2011-08-11 15:25:39,568 INFO org.apache.hadoop.yarn.webapp.WebApps: > > Registered webapp guice modules > > 2011-08-11 15:25:39,589 WARN > org.apache.hadoop.metrics2.impl.MetricsConfig: > > Cannot locate configuration: tried > > hadoop-metrics2-resourcemanager.properties,hadoop-metrics2.properties > > 2011-08-11 15:25:39,626 INFO > > org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot > period > > at 10 second(s). > > 2011-08-11 15:25:39,626 INFO > > org.apache.hadoop.metrics2.impl.MetricsSystemImpl: ResourceManager > metrics > > system started > > 2011-08-11 15:25:39,627 INFO > > org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Registered source > > QueueMetrics,q0=default > > 2011-08-11 15:25:39,627 INFO > > org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Registered source > > UgiMetrics > > 2011-08-11 15:25:39,628 INFO > org.apache.hadoop.yarn.service.AbstractService: > > Service:Dispatcher is started. > > 2011-08-11 15:25:39,629 INFO > org.apache.hadoop.yarn.service.AbstractService: > > > Service:org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.ContainerAllocationExpirer > > is started. > > 2011-08-11 15:25:39,629 INFO > org.apache.hadoop.yarn.service.AbstractService: > > Service:AMLivelinessMonitor is started. > > 2011-08-11 15:25:39,629 INFO > org.apache.hadoop.yarn.service.AbstractService: > > Service:org.apache.hadoop.yarn.server.resourcemanager.NodesListManager is > > started. > > 2011-08-11 15:25:39,629 INFO > org.apache.hadoop.yarn.service.AbstractService: > > > Service:org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher > > is started. > > 2011-08-11 15:25:39,629 INFO > org.apache.hadoop.yarn.service.AbstractService: > > Service:NMLivelinessMonitor is started. > > 2011-08-11 15:25:39,629 INFO > org.apache.hadoop.yarn.service.AbstractService: > > > Service:org.apache.hadoop.yarn.server.resourcemanager.ResourceTrackerService > > is started. > > 2011-08-11 15:25:39,630 INFO org.apache.hadoop.yarn.ipc.YarnRPC: Creating > > YarnRPC for null > > 2011-08-11 15:25:39,631 INFO org.apache.hadoop.yarn.ipc.HadoopYarnRPC: > > Creating a HadoopYarnProtoRpc server for protocol interface > > org.apache.hadoop.yarn.server.api.ResourceTracker with 10 handlers > > 2011-08-11 15:25:39,631 INFO org.apache.hadoop.yarn.ipc.HadoopYarnRPC: > > Configured SecurityInfo class name is > > org.apache.hadoop.yarn.server.RMNMSecurityInfoClass > > 2011-08-11 15:25:44,651 INFO org.apache.hadoop.ipc.Server: Starting > Socket > > Reader #1 for port 8025 > > 2011-08-11 15:25:44,663 INFO > > org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Registered source > > RpcActivityForPort8025 > > 2011-08-11 15:25:44,668 INFO > > org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Registered source > > RpcDetailedActivityForPort8025 > > 2011-08-11 15:25:44,670 ERROR > > org.apache.hadoop.yarn.service.CompositeService: Error starting services > > ResourceManager > > java.lang.NoSuchMethodError: org.apache.avro.ipc.Server.start()V > > at > > > org.apache.hadoop.yarn.server.resourcemanager.ResourceTrackerService.start(ResourceTrackerService.java:128) > > at > > > org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:68) > > at > > > org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.start(ResourceManager.java:392) > > at > > > org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:491) > > 2011-08-11 15:25:44,670 INFO > org.apache.hadoop.yarn.service.AbstractService: > > Service:NMLivelinessMonitor is stopped. > > 2011-08-11 15:25:44,671 INFO > > org.apache.hadoop.yarn.util.AbstractLivelinessMonitor: > NMLivelinessMonitor > > thread interrupted > > 2011-08-11 15:25:44,671 ERROR > > org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Returning, > > interrupted : java.lang.InterruptedException > > 2011-08-11 15:25:44,671 INFO > org.apache.hadoop.yarn.service.AbstractService: > > > Service:org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher > > is stopped. > > 2011-08-11 15:25:44,671 INFO > org.apache.hadoop.yarn.service.AbstractService: > > Service:org.apache.hadoop.yarn.server.resourcemanager.NodesListManager is > > stopped. > > 2011-08-11 15:25:44,671 INFO > org.apache.hadoop.yarn.service.AbstractService: > > Service:AMLivelinessMonitor is stopped. > > 2011-08-11 15:25:44,671 INFO > org.apache.hadoop.yarn.service.AbstractService: > > > Service:org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.ContainerAllocationExpirer > > is stopped. > > 2011-08-11 15:25:44,671 INFO > > org.apache.hadoop.yarn.util.AbstractLivelinessMonitor: > AMLivelinessMonitor > > thread interrupted > > 2011-08-11 15:25:44,671 INFO > > org.apache.hadoop.yarn.util.AbstractLivelinessMonitor: > > > org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.ContainerAllocationExpirer > > thread interrupted > > 2011-08-11 15:25:44,671 INFO > org.apache.hadoop.yarn.event.AsyncDispatcher: > > AsyncDispatcher thread interrupted > > java.lang.InterruptedException > > at > > > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:1961) > > at > > > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1996) > > at > > > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399) > > at > > > org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:69) > > at java.lang.Thread.run(Thread.java:662) > > 2011-08-11 15:25:44,672 INFO > org.apache.hadoop.yarn.service.AbstractService: > > Service:Dispatcher is stopped. > > 2011-08-11 15:25:44,672 ERROR > > org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error > > starting RM > > org.apache.hadoop.yarn.YarnException: Failed to Start ResourceManager > > at > > > org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:80) > > at > > > org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.start(ResourceManager.java:392) > > at > > > org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:491) > > Caused by: java.lang.NoSuchMethodError: > org.apache.avro.ipc.Server.start()V > > at > > > org.apache.hadoop.yarn.server.resourcemanager.ResourceTrackerService.start(ResourceTrackerService.java:128) > > at > > > org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:68) > > ... 2 more > > 2011-08-11 15:25:44,673 INFO org.mortbay.log: Stopped > > SelectChannelConnector@0.0.0.0:8088 > > 2011-08-11 15:25:44,775 INFO > > org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping > ResourceManager > > metrics system... > > 2011-08-11 15:25:44,776 INFO > > org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping metrics > source > > QueueMetrics,q0=default > > 2011-08-11 15:25:44,776 INFO > > org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping metrics > source > > UgiMetrics > > 2011-08-11 15:25:44,776 INFO > > org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping metrics > source > > RpcActivityForPort8025 > > 2011-08-11 15:25:44,776 INFO > > org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping metrics > source > > RpcDetailedActivityForPort8025 > > 2011-08-11 15:25:44,777 INFO > > org.apache.hadoop.metrics2.impl.MetricsSystemImpl: ResourceManager > metrics > > system stopped. > > 2011-08-11 15:25:44,777 INFO > > org.apache.hadoop.metrics2.impl.MetricsSystemImpl: ResourceManager > metrics > > system shutdown complete. > > ===================================================== > > On Thu, Aug 11, 2011 at 2:07 PM, Mahadev Konar <maha...@hortonworks.com> > > wrote: > >> > >> Matei, > >> Are you sure you are using the latest common from trunk? The > ClientCache > >> was recently added within the last few weeks. > >> Looks like its using some old version of hadoop common. > >> > >> Also the sl4j errors should be fixed in the latest MR-279 branch. > >> > >> thanks > >> mahadev > >> > >> On Aug 11, 2011, at 1:52 PM, Matei Zaharia wrote: > >> > >> > I get a similar error on Mac OS X. I've built YARN and extracted the > >> > tarball to a directory, but when I run bin/yarn-daemon.sh start > nodemanager, > >> > it prints the following to its log (apologies for the long trace): > >> > > >> > log4j:WARN No appenders could be found for logger > >> > (org.apache.hadoop.metrics2.impl.MetricsSystemImpl). > >> > log4j:WARN Please initialize the log4j system properly. > >> > log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfigfor > >> > more info. > >> > Exception in thread "main" org.apache.hadoop.yarn.YarnException: > Failed > >> > to Start org.apache.hadoop.yarn.server.nodemanager.NodeManager > >> > at > >> > > org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:80) > >> > at > >> > > org.apache.hadoop.yarn.server.nodemanager.NodeManager.start(NodeManager.java:146) > >> > at > >> > > org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:191) > >> > Caused by: org.apache.avro.AvroRuntimeException: > >> > org.apache.hadoop.yarn.YarnException: > >> > java.lang.reflect.InvocationTargetException > >> > at > >> > > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.start(NodeStatusUpdaterImpl.java:140) > >> > at > >> > > org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:68) > >> > ... 2 more > >> > Caused by: org.apache.hadoop.yarn.YarnException: > >> > java.lang.reflect.InvocationTargetException > >> > at > >> > > org.apache.hadoop.yarn.factories.impl.pb.RpcClientFactoryPBImpl.getClient(RpcClientFactoryPBImpl.java:70) > >> > at > >> > > org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC.getProxy(HadoopYarnProtoRPC.java:35) > >> > at > >> > > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.getRMClient(NodeStatusUpdaterImpl.java:158) > >> > at > >> > > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.registerWithRM(NodeStatusUpdaterImpl.java:163) > >> > at > >> > > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.start(NodeStatusUpdaterImpl.java:136) > >> > ... 3 more > >> > Caused by: java.lang.reflect.InvocationTargetException > >> > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > >> > Method) > >> > at > >> > > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) > >> > at > >> > > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) > >> > at > java.lang.reflect.Constructor.newInstance(Constructor.java:513) > >> > at > >> > > org.apache.hadoop.yarn.factories.impl.pb.RpcClientFactoryPBImpl.getClient(RpcClientFactoryPBImpl.java:67) > >> > ... 7 more > >> > Caused by: java.lang.NoClassDefFoundError: > >> > org/apache/hadoop/ipc/ClientCache > >> > at > >> > > org.apache.hadoop.yarn.ipc.ProtoOverHadoopRpcEngine.<clinit>(ProtoOverHadoopRpcEngine.java:63) > >> > at java.lang.Class.forName0(Native Method) > >> > at java.lang.Class.forName(Class.java:247) > >> > at > >> > > org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1108) > >> > at > >> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1160) > >> > at org.apache.hadoop.ipc.RPC.getProtocolEngine(RPC.java:94) > >> > at org.apache.hadoop.ipc.RPC.getProtocolProxy(RPC.java:422) > >> > at org.apache.hadoop.ipc.RPC.getProtocolProxy(RPC.java:368) > >> > at org.apache.hadoop.ipc.RPC.getProtocolProxy(RPC.java:333) > >> > at org.apache.hadoop.ipc.RPC.getProtocolProxy(RPC.java:461) > >> > at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:442) > >> > at > >> > > org.apache.hadoop.yarn.server.api.impl.pb.client.ResourceTrackerPBClientImpl.<init>(ResourceTrackerPBClientImpl.java:32) > >> > ... 12 more > >> > Caused by: java.lang.ClassNotFoundException: > >> > org.apache.hadoop.ipc.ClientCache > >> > at java.net.URLClassLoader$1.run(URLClassLoader.java:202) > >> > at java.security.AccessController.doPrivileged(Native Method) > >> > at java.net.URLClassLoader.findClass(URLClassLoader.java:190) > >> > at java.lang.ClassLoader.loadClass(ClassLoader.java:306) > >> > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) > >> > at java.lang.ClassLoader.loadClass(ClassLoader.java:247) > >> > ... 24 more > >> > > >> > In addition, if I try to start resourcemanager instead, it logs: > >> > > >> > SLF4J: Class path contains multiple SLF4J bindings. > >> > SLF4J: Found binding in > >> > > [jar:file:/Users/matei/workspace/MR-279/common/build/ivy/lib/Hadoop-Common/common/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] > >> > SLF4J: Found binding in > >> > > [jar:file:/Users/matei/workspace/MR-279-deploy/hadoop-mapreduce-1.0-SNAPSHOT/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] > >> > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an > >> > explanation. > >> > log4j:WARN No appenders could be found for logger > >> > (org.apache.hadoop.metrics2.lib.MutableMetricsFactory). > >> > log4j:WARN Please initialize the log4j system properly. > >> > log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfigfor > >> > more info. > >> > Exception in thread "main" java.lang.IllegalStateException: For this > >> > operation, current State must be STARTED instead of INITED > >> > at > >> > > org.apache.hadoop.yarn.service.AbstractService.ensureCurrentState(AbstractService.java:101) > >> > at > >> > > org.apache.hadoop.yarn.service.AbstractService.stop(AbstractService.java:69) > >> > at > >> > > org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher.stop(ApplicationMasterLauncher.java:90) > >> > at > >> > > org.apache.hadoop.yarn.service.CompositeService.stop(CompositeService.java:89) > >> > at > >> > > org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.stop(ResourceManager.java:423) > >> > at > >> > > org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:493) > >> > > >> > I don't know where the multiple SLF4J versions are coming from because > I > >> > checked out common, hdfs and mapreduce at the same time. > >> > > >> > Matei > >> > > >> > > >> > > >> > > >> > On Aug 11, 2011, at 2:01 AM, Vinod KV wrote: > >> > > >> >> > >> >> > >> >> Report YARN_* and HADOOP_* variables that you've set before starting > >> >> the daemons. > >> >> > >> >> Also run 'bin/yarn classpath' and post the output of that command > too. > >> >> If the output doesn't contain > yarn-server-resourcemanager-1.0-SNAPSHOT.jar, > >> >> you are missing something. > >> >> > >> >> +Vinod > >> >> > >> >> > >> >> On Thursday 11 August 2011 01:57 AM, Haoyuan Li wrote: > >> >>> Hi, > >> >>> > >> >>> When I ran ResourceManager or NodeManager as steps here > >> >>> > >> >>> > http://svn.apache.org/repos/asf/hadoop/common/branches/MR-279/mapreduce/INSTALL > >> >>> . > >> >>> It has main class can't be found exception... I attached the shell > >> >>> output > >> >>> here. Any help will be appreciated! > >> >>> > >> >>> Thanks, > >> >>> > >> >>> Haoyuan > >> >>> > >> >>> haoyuan@hya:~/hadoop/hadoop-mapreduce-1.0-SNAPSHOT$ > >> >>> ./bin/yarn-daemon.sh > >> >>> start resourcemanager > >> >>> starting resourcemanager, logging to > >> >>> > >> >>> > /home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../logs/yarn-haoyuan-resourcemanager-hya.out > >> >>> /usr/lib/jvm/java-6-sun/bin/java -Dproc_resourcemanager -Xmx1000m > >> >>> > >> >>> > -Dhadoop.log.dir=/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../logs > >> >>> > >> >>> > -Dyarn.log.dir=/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../logs > >> >>> -Dhadoop.log.file=yarn-haoyuan-resourcemanager-hya.log > >> >>> -Dyarn.log.file=yarn-haoyuan-resourcemanager-hya.log > -Dyarn.home.dir= > >> >>> -Dyarn.id.str=haoyuan -Dhadoop.root.logger=INFO,DRFA > >> >>> -Dyarn.root.logger=INFO,DRFA -Dyarn.policy.file=hadoop-policy.xml > >> >>> > >> >>> > -Dhadoop.log.dir=/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../logs > >> >>> > >> >>> > -Dyarn.log.dir=/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../logs > >> >>> -Dhadoop.log.file=yarn-haoyuan-resourcemanager-hya.log > >> >>> -Dyarn.log.file=yarn-haoyuan-resourcemanager-hya.log > >> >>> > >> >>> > -Dyarn.home.dir=/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/.. > >> >>> -Dhadoop.root.logger=INFO,DRFA -Dyarn.root.logger=INFO,DRFA > -classpath > >> >>> > >> >>> > /home/haoyuan/hadoop/conf:/home/haoyuan/hadoop/conf:/usr/lib/jvm/java-6-sun/lib/tools.jar:/home/haoyuan/hadoop/trunk/common/hadoop-*.jar:/home/haoyuan/hadoop/trunk/common/lib/*.jar:/home/haoyuan/hadoop/trunk/common/share/hadoop/common/*.jar:/home/haoyuan/hadoop/trunk/common/share/hadoop/common/lib/*.jar:/home/haoyuan/hadoop/trunk/common/share/hadoop/hdfs/*.jar:/home/haoyuan/hadoop/trunk/hdfs/hadoop-*.jar:/home/haoyuan/hadoop/trunk/hdfs/lib/*.jar:/home/haoyuan/hadoop/trunk/hdfs/build/classes:/home/haoyuan/hadoop/trunk/hdfs/hadoop-*.jar:/home/haoyuan/hadoop/trunk/hdfs/lib/*.jar:/home/haoyuan/hadoop/trunk/mapreduce/build/classes:/home/haoyuan/hadoop/trunk/mapreduce/build:/home/haoyuan/hadoop/trunk/mapreduce/build/test/classes:/home/haoyuan/hadoop/trunk/mapreduce/build/tools:/home/haoyuan/hadoop/trunk/mapreduce/lib/*.jar:/home/haoyuan/hadoop/trunk/mapreduce/*.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/aopalliance-1.0.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/asm-3.2.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/aspectjrt-1.6.5.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/avro-1.3.2.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/avro-1.4.1.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/clover-3.0.2.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/commons-beanutils-1.7.0.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/commons-beanutils-core-1.8.0.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/commons-cli-1.2.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/commons-codec-1.4.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/commons-collections-3.2.1.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/commons-configuration-1.6.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/commons-digester-1.8.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/commons-httpclient-3.1.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/commons-lang-2.5.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/commons-logging-1.0.4.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/commons-logging-api-1.0.4.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/commons-math-2.1.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/commons-net-1.4.1.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/core-3.1.1.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/guava-r09.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/guice-2.0.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/guice-servlet-2.0.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/hadoop-annotations-0.23.0-SNAPSHOT.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/jackson-core-asl-1.4.2.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/jackson-mapper-asl-1.4.2.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/jdiff-1.0.9.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/jets3t-0.6.1.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/jetty-6.1.26.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/jetty-util-6.1.26.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/jsp-api-2.1.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/junit-4.8.2.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/kfs-0.3.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/log4j-1.2.15.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/netty-3.2.3.Final.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/oro-2.0.8.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/paranamer-2.2.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/paranamer-ant-2.2.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/paranamer-generator-2.2.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/protobuf-java-2.4.0a.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/qdox-1.10.1.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/servlet-api-2.5.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/slf4j-api-1.6.1.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/slf4j-log4j12-1.6.1.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/xmlenc-0.52.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../modules/hadoop-mapreduce-client-app-1.0-SNAPSHOT.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../modules/hadoop-mapreduce-client-common-1.0-SNAPSHOT.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../modules/hadoop-mapreduce-client-core-1.0-SNAPSHOT.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../modules/hadoop-mapreduce-client-hs-1.0-SNAPSHOT.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../modules/hadoop-mapreduce-client-jobclient-1.0-SNAPSHOT.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../modules/hadoop-mapreduce-client-shuffle-1.0-SNAPSHOT.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../modules/yarn-api-1.0-SNAPSHOT.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../modules/yarn-common-1.0-SNAPSHOT.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../modules/yarn-server-common-1.0-SNAPSHOT.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../modules/yarn-server-nodemanager-1.0-SNAPSHOT.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../modules/yarn-server-resourcemanager-1.0-SNAPSHOT.jar:/home/haoyuan/hadoop/conf/rm-config/log4j.properties > >> >>> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager > >> >>> Exception in thread "main" java.lang.NoClassDefFoundError: > >> >>> org/apache/hadoop/conf/Configuration > >> >>> Caused by: java.lang.ClassNotFoundException: > >> >>> org.apache.hadoop.conf.Configuration > >> >>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202) > >> >>> at java.security.AccessController.doPrivileged(Native Method) > >> >>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190) > >> >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:306) > >> >>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) > >> >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:247) > >> >>> Could not find the main class: > >> >>> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager. > >> >>> Program will > >> >>> exit. > >> >>> > >> >> > >> >> > >> > > >> > > > > >
