Comment all the IPs in /etc/hosts first except localhost or 127.0.0.1 as i have given below. Then, give in conf/hadoop-site.xml fs.default.name parameter value is hdfs://hdfs://[Master Node IP]:9000/ mapred.job.tracker parameter value is hdfs://[Job Tracker IP]:9001 Make sure you have enabled ssh and sshd running without password.
lazikid wrote: > > I need some help with setting up a Hadoop cluster. The datanode on the > slave is not coming up throwing java.net.NoRouteToHostException: No route > to host. Please see the details below. > > I have a centos master and a fedora slave. Both have Java 6 and Hadoop > 0.20.1. I instaleed hadoop under /opt in both machines. > > The machines can ping one another using hostname and ip. They can also > make password-less ssh connection to one another using both hostname and > ip. > > This is what I have in the /etc/hosts file for the master : > > ************************************************ > 127.0.0.1 localhost.localdomain localhost > #192.168.1.125 centos > #192.168.1.102 fedora > ::1 localhost6.localdomain6 localhost6 > ************************************************ > > This is what I have in the /etc/hosts for the slave > > ************************************************ > 127.0.0.1 localhost.localdomain localhost > #192.168.1.104 ubuntu64 > #192.168.1.102 fedora > #192.168.1.125 centos > ::1 localhost localhost.localdomain localhost6 > localhost6.localdomain6 > ************************************************ > > > > These are my configuration files : > > 1. core-site.xml > > <configuration> > <property> > <name>fs.default.name</name> > <value>hdfs://centos:54310</value> > <description>determine the host, port, etc. for a > filesystem.</description> > </property> > </configuration> > > 2. mapred-site.xml > > <configuration> > <property> > <name>mapred.job.tracker</name> > <value>hdfs://centos:54311</value> > <description>....</description> > </property> > </configuration> > > 3. masters > > centos > > 4. slaves > > fedora > > > I issued bin/start-dfs.sh from the master(centos) machine and this is what > I see : > > starting namenode, logging to xxxxxxxxxxx > fedora: starting datanode, logging to xxxxxxx > centos: starting secondarynamenode, logging to xxxxxx > > When I checked the slave(fedora) logs, this what I see : > > ************************************************************************* > STARTUP_MSG: Starting DataNode > STARTUP_MSG: host = fedora/192.168.1.102 > STARTUP_MSG: args = [] > STARTUP_MSG: version = 0.20.1 > STARTUP_MSG: build = > http://svn.apache.org/repos/asf/hadoop/common/tags/release-0.20.1-rc1 -r > 810220; compiled by 'oom' on Tue Sep 1 20:55:56 UTC > > 2009 > ************************************************************/ > 2009-10-17 17:05:43,385 INFO org.apache.hadoop.ipc.Client: Retrying > connect to server: centos/192.168.1.125:54310. Already tried 0 time(s). > 2009-10-17 17:05:44,387 INFO org.apache.hadoop.ipc.Client: Retrying > connect to server: centos/192.168.1.125:54310. Already tried 1 time(s). > 2009-10-17 17:05:45,389 INFO org.apache.hadoop.ipc.Client: Retrying > connect to server: centos/192.168.1.125:54310. Already tried 2 time(s). > 2009-10-17 17:05:46,390 INFO org.apache.hadoop.ipc.Client: Retrying > connect to server: centos/192.168.1.125:54310. Already tried 3 time(s). > 2009-10-17 17:05:47,388 INFO org.apache.hadoop.ipc.Client: Retrying > connect to server: centos/192.168.1.125:54310. Already tried 4 time(s). > 2009-10-17 17:05:48,390 INFO org.apache.hadoop.ipc.Client: Retrying > connect to server: centos/192.168.1.125:54310. Already tried 5 time(s). > 2009-10-17 17:05:49,393 INFO org.apache.hadoop.ipc.Client: Retrying > connect to server: centos/192.168.1.125:54310. Already tried 6 time(s). > 2009-10-17 17:05:50,394 INFO org.apache.hadoop.ipc.Client: Retrying > connect to server: centos/192.168.1.125:54310. Already tried 7 time(s). > 2009-10-17 17:05:51,395 INFO org.apache.hadoop.ipc.Client: Retrying > connect to server: centos/192.168.1.125:54310. Already tried 8 time(s). > 2009-10-17 17:05:52,398 INFO org.apache.hadoop.ipc.Client: Retrying > connect to server: centos/192.168.1.125:54310. Already tried 9 time(s). > 2009-10-17 17:05:52,404 ERROR > org.apache.hadoop.hdfs.server.datanode.DataNode: java.io.IOException: Call > to centos/192.168.1.125:54310 failed on local > > exception: java.net.NoRouteToHostException: No route to host > at org.apache.hadoop.ipc.Client.wrapException(Client.java:774) > at org.apache.hadoop.ipc.Client.call(Client.java:742) > at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220) > at $Proxy4.getProtocolVersion(Unknown Source) > at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:359) > at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:346) > at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:383) > at org.apache.hadoop.ipc.RPC.waitForProxy(RPC.java:314) > at org.apache.hadoop.ipc.RPC.waitForProxy(RPC.java:291) > at > org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:269) > at > org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:216) > at > org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1283) > at > org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1238) > at > org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1246) > at > org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1368) > Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) > at > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574) > at > org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206) > at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:404) > at > org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:304) > at org.apache.hadoop.ipc.Client$Connection.access$1700(Client.java:176) > at org.apache.hadoop.ipc.Client.getConnection(Client.java:859) > at org.apache.hadoop.ipc.Client.call(Client.java:719) > ... 13 more > > 2009-10-17 17:05:52,407 INFO > org.apache.hadoop.hdfs.server.datanode.DataNode: SHUTDOWN_MSG: > /************************************************************ > SHUTDOWN_MSG: Shutting down DataNode at fedora/192.168.1.102 > ************************************************************/ > > ************************************************************************* > > > > > > The log on the master shows : > > 2009-10-17 17:05:39,352 INFO > org.apache.hadoop.hdfs.server.namenode.NameNode: STARTUP_MSG: > /************************************************************ > STARTUP_MSG: Starting NameNode > STARTUP_MSG: host = centos/192.168.1.125 > STARTUP_MSG: args = [] > STARTUP_MSG: version = 0.20.1 > STARTUP_MSG: build = > http://svn.apache.org/repos/asf/hadoop/common/tags/release-0.20.1-rc1 -r > 810220; compiled by 'oom' on Tue Sep 1 20:55:56 UTC > > 2009 > ************************************************************/ > 2009-10-17 17:05:39,615 INFO org.apache.hadoop.ipc.metrics.RpcMetrics: > Initializing RPC Metrics with hostName=NameNode, port=54310 > 2009-10-17 17:05:39,620 INFO > org.apache.hadoop.hdfs.server.namenode.NameNode: Namenode up at: > centos/192.168.1.125:54310 > 2009-10-17 17:05:39,622 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: > Initializing JVM Metrics with processName=NameNode, sessionId=null > 2009-10-17 17:05:39,624 INFO > org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics: > Initializing NameNodeMeterics using context > > object:org.apache.hadoop.metrics.spi.NullContext > 2009-10-17 17:05:39,779 INFO > org.apache.hadoop.hdfs.server.namenode.FSNamesystem: fsOwner=joe,joe > 2009-10-17 17:05:39,780 INFO > org.apache.hadoop.hdfs.server.namenode.FSNamesystem: supergroup=supergroup > 2009-10-17 17:05:39,781 INFO > org.apache.hadoop.hdfs.server.namenode.FSNamesystem: > isPermissionEnabled=true > 2009-10-17 17:05:39,877 INFO > org.apache.hadoop.hdfs.server.namenode.metrics.FSNamesystemMetrics: > Initializing FSNamesystemMetrics using context > > object:org.apache.hadoop.metrics.spi.NullContext > 2009-10-17 17:05:39,883 INFO > org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Registered > FSNamesystemStatusMBean > 2009-10-17 17:05:39,929 INFO org.apache.hadoop.hdfs.server.common.Storage: > Number of files = 1 > 2009-10-17 17:05:39,933 INFO org.apache.hadoop.hdfs.server.common.Storage: > Number of files under construction = 0 > 2009-10-17 17:05:39,933 INFO org.apache.hadoop.hdfs.server.common.Storage: > Image file of size 97 loaded in 0 seconds. > 2009-10-17 17:05:39,934 INFO org.apache.hadoop.hdfs.server.common.Storage: > Edits file /tmp/hadoop-joe/dfs/name/current/edits of size 4 edits # 0 > > loaded in 0 seconds. > 2009-10-17 17:05:39,972 INFO org.apache.hadoop.hdfs.server.common.Storage: > Image file of size 97 saved in 0 seconds. > 2009-10-17 17:05:40,130 INFO > org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Finished loading > FSImage in 469 msecs > 2009-10-17 17:05:40,131 INFO > org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Total number of > blocks = 0 > 2009-10-17 17:05:40,131 INFO > org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Number of invalid > blocks = 0 > 2009-10-17 17:05:40,131 INFO > org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Number of > under-replicated blocks = 0 > 2009-10-17 17:05:40,132 INFO > org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Number of > over-replicated blocks = 0 > 2009-10-17 17:05:40,132 INFO org.apache.hadoop.hdfs.StateChange: STATE* > Leaving safe mode after 0 secs. > 2009-10-17 17:05:40,132 INFO org.apache.hadoop.hdfs.StateChange: STATE* > Network topology has 0 racks and 0 datanodes > 2009-10-17 17:05:40,132 INFO org.apache.hadoop.hdfs.StateChange: STATE* > UnderReplicatedBlocks has 0 blocks > 2009-10-17 17:05:40,320 INFO org.mortbay.log: Logging to > org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via > org.mortbay.log.Slf4jLog > 2009-10-17 17:05:40,518 INFO org.apache.hadoop.http.HttpServer: Port > returned by webServer.getConnectors()[0].getLocalPort() before open() is > -1. > > Opening the listener on 50070 > 2009-10-17 17:05:40,520 INFO org.apache.hadoop.http.HttpServer: > listener.getLocalPort() returned 50070 > webServer.getConnectors()[0].getLocalPort() > > returned 50070 > 2009-10-17 17:05:40,520 INFO org.apache.hadoop.http.HttpServer: Jetty > bound to port 50070 > 2009-10-17 17:05:40,520 INFO org.mortbay.log: jetty-6.1.14 > 2009-10-17 17:05:46,612 INFO org.mortbay.log: Started > [email protected]:50070 > 2009-10-17 17:05:46,612 INFO > org.apache.hadoop.hdfs.server.namenode.NameNode: Web-server up at: > 0.0.0.0:50070 > 2009-10-17 17:05:46,613 INFO org.apache.hadoop.ipc.Server: IPC Server > Responder: starting > 2009-10-17 17:05:46,615 INFO org.apache.hadoop.ipc.Server: IPC Server > listener on 54310: starting > 2009-10-17 17:05:46,629 INFO org.apache.hadoop.ipc.Server: IPC Server > handler 0 on 54310: starting > 2009-10-17 17:05:46,634 INFO org.apache.hadoop.ipc.Server: IPC Server > handler 1 on 54310: starting > 2009-10-17 17:05:46,636 INFO org.apache.hadoop.ipc.Server: IPC Server > handler 2 on 54310: starting > 2009-10-17 17:05:46,637 INFO org.apache.hadoop.ipc.Server: IPC Server > handler 4 on 54310: starting > 2009-10-17 17:05:46,641 INFO org.apache.hadoop.ipc.Server: IPC Server > handler 6 on 54310: starting > 2009-10-17 17:05:46,642 INFO org.apache.hadoop.ipc.Server: IPC Server > handler 7 on 54310: starting > 2009-10-17 17:05:46,643 INFO org.apache.hadoop.ipc.Server: IPC Server > handler 8 on 54310: starting > 2009-10-17 17:05:46,643 INFO org.apache.hadoop.ipc.Server: IPC Server > handler 3 on 54310: starting > 2009-10-17 17:05:46,900 INFO org.apache.hadoop.ipc.Server: IPC Server > handler 5 on 54310: starting > 2009-10-17 17:05:46,916 INFO org.apache.hadoop.ipc.Server: IPC Server > handler 9 on 54310: starting > 2009-10-17 17:11:10,967 INFO > org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Roll Edit Log from > 192.168.1.125 > 2009-10-17 17:11:10,968 INFO > org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Number of > transactions: 0 Total time for transactions(ms): 0Number of > > transactions batched in Syncs: 0 Number of syncs: 0 SyncTimes(ms): 0 > 2009-10-17 17:11:11,518 INFO > org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Roll FSImage from > 192.168.1.125 > 2009-10-17 17:11:11,519 INFO > org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Number of > transactions: 0 Total time for transactions(ms): 0Number of > > transactions batched in Syncs: 0 Number of syncs: 1 SyncTimes(ms): 2 > > > > -- View this message in context: http://www.nabble.com/Datanode-Throwing-NoRouteToHostException-tp25946798p25986468.html Sent from the Hadoop core-user mailing list archive at Nabble.com.
