I got it solved by recompiling nutch using the new hadoop-0.2-dev.jar
from the nightly instead
of using the hadoop-0.1.1.jar originally in nutch/trunk/libs
-Corrado
zzcgiacomini wrote:
Hello everybody,
I am new to Nutch, I just start evaluating it a couple of days ago....
I have installed the yestarday nightlty build on two machines for
testing, one is running as a master
the second one is my only slave right now. The ssh on the two machines
has been configured properly so I can login with no password between them
On the Master machine I have started to crawl.
Hadoop the DFS is working fine I can see from the logs that the slave
machines is receiving blocks from the master.
My problem is the tasktraker on the slave machine. When started it get
connected to the jobtracker on the master machine
but as soon as this late one seams to dispatch tasks to the slave then
I get the following error (see log below)
From the code in TaskTracker.java:756 I can not deduce much more that
is a FSError
Any helps ?
060428 120134 parsing
jar:file:/ke/disk10/nutch-0.8-dev/lib/hadoop-0.1.1.jar!/hadoop-default.xml
060428 120134 parsing file:/ke/disk10/nutch-0.8-dev/conf/hadoop-site.xml
060428 120134 Starting tracker tracker_61301
060428 120134 parsing
jar:file:/ke/disk10/nutch-0.8-dev/lib/hadoop-0.1.1.jar!/hadoop-default.xml
060428 120134 parsing
jar:file:/ke/disk10/nutch-0.8-dev/lib/hadoop-0.1.1.jar!/mapred-default.xml
060428 120134 parsing file:/ke/disk10/nutch-0.8-dev/conf/hadoop-site.xml
060428 120134 Server listener on port 50050: starting
060428 120134 Server handler 0 on 50050: starting
060428 120134 Server handler 1 on 50050: starting
060428 120134 Server listener on port 50040: starting
060428 120134 Server handler 0 on 50040: starting
060428 120134 parsing
jar:file:/ke/disk10/nutch-0.8-dev/lib/hadoop-0.1.1.jar!/hadoop-default.xml
060428 120134 parsing
jar:file:/ke/disk10/nutch-0.8-dev/lib/hadoop-0.1.1.jar!/mapred-default.xml
060428 120134 parsing file:/ke/disk10/nutch-0.8-dev/conf/hadoop-site.xml
060428 120134 Server handler 1 on 50040: starting
060428 120134 Client connection to 10.234.57.38:9011: starting
060428 120304 parsing
jar:file:/ke/disk10/nutch-0.8-dev/lib/hadoop-0.1.1.jar!/hadoop-default.xml
060428 120304 parsing
jar:file:/ke/disk10/nutch-0.8-dev/lib/hadoop-0.1.1.jar!/mapred-default.xml
060428 120304 parsing file:/ke/disk10/nutch-0.8-dev/conf/hadoop-site.xml
060428 120304 Lost connection to JobTracker
[bas025.dev.gen01.ke.wanadoo.fr/10.234.57.38:9011]. Retrying...
java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
at
java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
at java.net.Socket.connect(Socket.java:507)
at java.net.Socket.connect(Socket.java:457)
at java.net.Socket.<init>(Socket.java:365)
at java.net.Socket.<init>(Socket.java:207)
at org.apache.hadoop.ipc.Client$Connection.<init>(Client.java:114)
at org.apache.hadoop.ipc.Client.getConnection(Client.java:352)
at org.apache.hadoop.ipc.Client.call(Client.java:290)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:141)
at org.apache.hadoop.dfs.$Proxy1.isDir(Unknown Source)
at org.apache.hadoop.dfs.DFSClient.isDirectory(DFSClient.java:127)
at
org.apache.hadoop.dfs.DistributedFileSystem.isDirectory(DistributedFileSystem.java:108)
at
org.apache.hadoop.dfs.DistributedFileSystem.copyToLocalFile(DistributedFileSystem.java:216)
at
org.apache.hadoop.mapred.TaskTracker$TaskInProgress.localizeTask(TaskTracker.java:397)
at
org.apache.hadoop.mapred.TaskTracker$TaskInProgress.<init>(TaskTracker.java:383)
at
org.apache.hadoop.mapred.TaskTracker.offerService(TaskTracker.java:270)
at org.apache.hadoop.mapred.TaskTracker.run(TaskTracker.java:336)
at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.jav
a:756)
060428 120309 parsing
jar:file:/ke/disk10/nutch-0.8-dev/lib/hadoop-0.1.1.jar!/hadoop-default.xml
....
....
