I need some help with setting up a Hadoop cluster. The datanode on the slave
is not coming up throwing java.net.NoRouteToHostException: No route to host.
Please see the details below.

I have a centos master and a fedora slave. Both have Java 6 and Hadoop
0.20.1. I instaleed hadoop under /opt in both machines.

The machines can ping one another using hostname and ip. They can also make
password-less ssh connection to one another using both hostname and ip.

This is what I have in the /etc/hosts file for the master :

************************************************
127.0.0.1       localhost.localdomain localhost
192.168.1.125   centos
192.168.1.102   fedora
::1             localhost6.localdomain6 localhost6
************************************************

This is what I have in the /etc/hosts for the slave

************************************************
127.0.0.1       localhost.localdomain localhost
192.168.1.104   ubuntu64
192.168.1.102   fedora
192.168.1.125   centos
::1         localhost localhost.localdomain localhost6
localhost6.localdomain6
************************************************



These are my configuration files :

1. core-site.xml 

<configuration>
<property>
  <name>fs.default.name</name>
  <value>hdfs://centos:54310</value>
  <description>determine the host, port, etc. for a
filesystem.</description>
</property>
</configuration>

2. mapred-site.xml

<configuration>
<property>
  <name>mapred.job.tracker</name>
  <value>hdfs://centos:54311</value>
  <description>....</description>
</property>
</configuration>

3. masters
  
   centos

4. slaves

   fedora


I issued bin/start-dfs.sh from the master(centos) machine and this is what I
see :

starting namenode, logging to xxxxxxxxxxx
fedora: starting datanode, logging to xxxxxxx
centos: starting secondarynamenode, logging to xxxxxx

When I checked the slave(fedora) logs, this what I see :

*************************************************************************
STARTUP_MSG: Starting DataNode
STARTUP_MSG:   host = fedora/192.168.1.102
STARTUP_MSG:   args = []
STARTUP_MSG:   version = 0.20.1
STARTUP_MSG:   build =
http://svn.apache.org/repos/asf/hadoop/common/tags/release-0.20.1-rc1 -r
810220; compiled by 'oom' on Tue Sep  1 20:55:56 UTC 

2009
************************************************************/
2009-10-17 17:05:43,385 INFO org.apache.hadoop.ipc.Client: Retrying connect
to server: centos/192.168.1.125:54310. Already tried 0 time(s).
2009-10-17 17:05:44,387 INFO org.apache.hadoop.ipc.Client: Retrying connect
to server: centos/192.168.1.125:54310. Already tried 1 time(s).
2009-10-17 17:05:45,389 INFO org.apache.hadoop.ipc.Client: Retrying connect
to server: centos/192.168.1.125:54310. Already tried 2 time(s).
2009-10-17 17:05:46,390 INFO org.apache.hadoop.ipc.Client: Retrying connect
to server: centos/192.168.1.125:54310. Already tried 3 time(s).
2009-10-17 17:05:47,388 INFO org.apache.hadoop.ipc.Client: Retrying connect
to server: centos/192.168.1.125:54310. Already tried 4 time(s).
2009-10-17 17:05:48,390 INFO org.apache.hadoop.ipc.Client: Retrying connect
to server: centos/192.168.1.125:54310. Already tried 5 time(s).
2009-10-17 17:05:49,393 INFO org.apache.hadoop.ipc.Client: Retrying connect
to server: centos/192.168.1.125:54310. Already tried 6 time(s).
2009-10-17 17:05:50,394 INFO org.apache.hadoop.ipc.Client: Retrying connect
to server: centos/192.168.1.125:54310. Already tried 7 time(s).
2009-10-17 17:05:51,395 INFO org.apache.hadoop.ipc.Client: Retrying connect
to server: centos/192.168.1.125:54310. Already tried 8 time(s).
2009-10-17 17:05:52,398 INFO org.apache.hadoop.ipc.Client: Retrying connect
to server: centos/192.168.1.125:54310. Already tried 9 time(s).
2009-10-17 17:05:52,404 ERROR
org.apache.hadoop.hdfs.server.datanode.DataNode: java.io.IOException: Call
to centos/192.168.1.125:54310 failed on local 

exception: java.net.NoRouteToHostException: No route to host
        at org.apache.hadoop.ipc.Client.wrapException(Client.java:774)
        at org.apache.hadoop.ipc.Client.call(Client.java:742)
        at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220)
        at $Proxy4.getProtocolVersion(Unknown Source)
        at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:359)
        at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:346)
        at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:383)
        at org.apache.hadoop.ipc.RPC.waitForProxy(RPC.java:314)
        at org.apache.hadoop.ipc.RPC.waitForProxy(RPC.java:291)
        at
org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:269)
        at
org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:216)
        at
org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1283)
        at
org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1238)
        at
org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1246)
        at 
org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1368)
Caused by: java.net.NoRouteToHostException: No route to host
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)
        at
org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
        at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:404)
        at 
org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:304)
        at org.apache.hadoop.ipc.Client$Connection.access$1700(Client.java:176)
        at org.apache.hadoop.ipc.Client.getConnection(Client.java:859)
        at org.apache.hadoop.ipc.Client.call(Client.java:719)
        ... 13 more

2009-10-17 17:05:52,407 INFO
org.apache.hadoop.hdfs.server.datanode.DataNode: SHUTDOWN_MSG: 
/************************************************************
SHUTDOWN_MSG: Shutting down DataNode at fedora/192.168.1.102
************************************************************/

*************************************************************************





The log on the master shows :

2009-10-17 17:05:39,352 INFO
org.apache.hadoop.hdfs.server.namenode.NameNode: STARTUP_MSG: 
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG:   host = centos/192.168.1.125
STARTUP_MSG:   args = []
STARTUP_MSG:   version = 0.20.1
STARTUP_MSG:   build =
http://svn.apache.org/repos/asf/hadoop/common/tags/release-0.20.1-rc1 -r
810220; compiled by 'oom' on Tue Sep  1 20:55:56 UTC 

2009
************************************************************/
2009-10-17 17:05:39,615 INFO org.apache.hadoop.ipc.metrics.RpcMetrics:
Initializing RPC Metrics with hostName=NameNode, port=54310
2009-10-17 17:05:39,620 INFO
org.apache.hadoop.hdfs.server.namenode.NameNode: Namenode up at:
centos/192.168.1.125:54310
2009-10-17 17:05:39,622 INFO org.apache.hadoop.metrics.jvm.JvmMetrics:
Initializing JVM Metrics with processName=NameNode, sessionId=null
2009-10-17 17:05:39,624 INFO
org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics: Initializing
NameNodeMeterics using context 

object:org.apache.hadoop.metrics.spi.NullContext
2009-10-17 17:05:39,779 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: fsOwner=joe,joe
2009-10-17 17:05:39,780 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: supergroup=supergroup
2009-10-17 17:05:39,781 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem:
isPermissionEnabled=true
2009-10-17 17:05:39,877 INFO
org.apache.hadoop.hdfs.server.namenode.metrics.FSNamesystemMetrics:
Initializing FSNamesystemMetrics using context 

object:org.apache.hadoop.metrics.spi.NullContext
2009-10-17 17:05:39,883 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Registered
FSNamesystemStatusMBean
2009-10-17 17:05:39,929 INFO org.apache.hadoop.hdfs.server.common.Storage:
Number of files = 1
2009-10-17 17:05:39,933 INFO org.apache.hadoop.hdfs.server.common.Storage:
Number of files under construction = 0
2009-10-17 17:05:39,933 INFO org.apache.hadoop.hdfs.server.common.Storage:
Image file of size 97 loaded in 0 seconds.
2009-10-17 17:05:39,934 INFO org.apache.hadoop.hdfs.server.common.Storage:
Edits file /tmp/hadoop-joe/dfs/name/current/edits of size 4 edits # 0 

loaded in 0 seconds.
2009-10-17 17:05:39,972 INFO org.apache.hadoop.hdfs.server.common.Storage:
Image file of size 97 saved in 0 seconds.
2009-10-17 17:05:40,130 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Finished loading
FSImage in 469 msecs
2009-10-17 17:05:40,131 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Total number of blocks
= 0
2009-10-17 17:05:40,131 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Number of invalid
blocks = 0
2009-10-17 17:05:40,131 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Number of
under-replicated blocks = 0
2009-10-17 17:05:40,132 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Number of 
over-replicated blocks = 0
2009-10-17 17:05:40,132 INFO org.apache.hadoop.hdfs.StateChange: STATE*
Leaving safe mode after 0 secs.
2009-10-17 17:05:40,132 INFO org.apache.hadoop.hdfs.StateChange: STATE*
Network topology has 0 racks and 0 datanodes
2009-10-17 17:05:40,132 INFO org.apache.hadoop.hdfs.StateChange: STATE*
UnderReplicatedBlocks has 0 blocks
2009-10-17 17:05:40,320 INFO org.mortbay.log: Logging to
org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via
org.mortbay.log.Slf4jLog
2009-10-17 17:05:40,518 INFO org.apache.hadoop.http.HttpServer: Port
returned by webServer.getConnectors()[0].getLocalPort() before open() is -1. 

Opening the listener on 50070
2009-10-17 17:05:40,520 INFO org.apache.hadoop.http.HttpServer:
listener.getLocalPort() returned 50070
webServer.getConnectors()[0].getLocalPort() 

returned 50070
2009-10-17 17:05:40,520 INFO org.apache.hadoop.http.HttpServer: Jetty bound
to port 50070
2009-10-17 17:05:40,520 INFO org.mortbay.log: jetty-6.1.14
2009-10-17 17:05:46,612 INFO org.mortbay.log: Started
[email protected]:50070
2009-10-17 17:05:46,612 INFO
org.apache.hadoop.hdfs.server.namenode.NameNode: Web-server up at:
0.0.0.0:50070
2009-10-17 17:05:46,613 INFO org.apache.hadoop.ipc.Server: IPC Server
Responder: starting
2009-10-17 17:05:46,615 INFO org.apache.hadoop.ipc.Server: IPC Server
listener on 54310: starting
2009-10-17 17:05:46,629 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 0 on 54310: starting
2009-10-17 17:05:46,634 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 1 on 54310: starting
2009-10-17 17:05:46,636 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 2 on 54310: starting
2009-10-17 17:05:46,637 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 4 on 54310: starting
2009-10-17 17:05:46,641 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 6 on 54310: starting
2009-10-17 17:05:46,642 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 7 on 54310: starting
2009-10-17 17:05:46,643 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 8 on 54310: starting
2009-10-17 17:05:46,643 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 3 on 54310: starting
2009-10-17 17:05:46,900 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 5 on 54310: starting
2009-10-17 17:05:46,916 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 9 on 54310: starting
2009-10-17 17:11:10,967 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Roll Edit Log from
192.168.1.125
2009-10-17 17:11:10,968 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Number of transactions:
0 Total time for transactions(ms): 0Number of 

transactions batched in Syncs: 0 Number of syncs: 0 SyncTimes(ms): 0 
2009-10-17 17:11:11,518 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Roll FSImage from
192.168.1.125
2009-10-17 17:11:11,519 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Number of transactions:
0 Total time for transactions(ms): 0Number of 

transactions batched in Syncs: 0 Number of syncs: 1 SyncTimes(ms): 2 


   
-- 
View this message in context: 
http://www.nabble.com/Datanode-Throwing-NoRouteToHostException-tp25946798p25946798.html
Sent from the Hadoop core-user mailing list archive at Nabble.com.

Reply via email to