Hello!
I want to test NDFS on my nutch installation, but I have some problem.
I have started from wiki, where is quick demo for NDFS:
http://wiki.apache.org/nutch/NutchDistributedFileSystem
On "$ nutch ndfs -put local_file /test/testfile"(or ./nutch admin db
-create and etc.) I always have exception "Could not obtain new output block":
=======================================================================
050830 061956 Waiting to find target node
Exception in thread "main" java.io.IOException: Could not obtain new
output block for file /test/testfile
at
org.apache.nutch.ndfs.NDFSClient$NameNodeCaller.getNewOutputBlock(NDFSClient.java:921)
at
org.apache.nutch.ndfs.NDFSClient$NDFSOutputStream.nextBlockOutputStream(NDFSClient.java:616)
at
org.apache.nutch.ndfs.NDFSClient$NDFSOutputStream.<init>(NDFSClient.java:597)
at org.apache.nutch.ndfs.NDFSClient.create(NDFSClient.java:85)
at org.apache.nutch.fs.NDFSFileSystem.create(NDFSFileSystem.java:76)
at org.apache.nutch.fs.NDFSFileSystem.create(NDFSFileSystem.java:71)
at org.apache.nutch.io.SequenceFile$Writer.<init>(SequenceFile.java:80)
at org.apache.nutch.io.MapFile$Writer.<init>(MapFile.java:94)
at org.apache.nutch.db.WebDBWriter.<init>(WebDBWriter.java:1507)
at org.apache.nutch.db.WebDBWriter.createWebDB(WebDBWriter.java:1438)
at org.apache.nutch.tools.WebDBAdminTool.main(WebDBAdminTool.java:172)
=======================================================================
On namenode I see something like this:
=======================================================================
050830 061445 Pending transfer from server.domain.com:7000 to 3 destinations
050830 061447 Renewed lease [Lease. Holder: NDFSClient_-1094164187, heldlocks:
1, pendingcreates: 1]
050830 061448 Pending transfer from server.domain.com:7000 to 3 destinations
050830 061451 Pending transfer from server.domain.com:7000 to 3 destinations
050830 061454 Pending transfer from server.domain.com:7000 to 3 destinations
050830 061455 Renewed lease [Lease. Holder: NDFSClient_-1094164187, heldlocks:
1, pendingcreates: 1]
050830 061457 Pending transfer from server.domain.com:7000 to 3 destinations
050830 061500 Pending transfer from server.domain.com:7000 to 3 destinations
050830 061503 Pending transfer from server.domain.com:7000 to 3 destinations
050830 061503 Renewed lease [Lease. Holder: NDFSClient_-1094164187, heldlocks:
1, pendingcreates: 1]
=======================================================================
But if I run datanode and namenode on the same server - all's ok!
On "$ nutch ndfs -report" I see list of my datanodes, but these
datanodes defined by external hostname. I think that namenode tries
to connect to datanodes by this NOT LOCAL hostnames. It is impossible
because firewall not allow incoming connection from external network
interfaces to this port(7000).
It's right? The error can be generated in this case?
So, can you tell me, please, what I can make to define namenode for use
local interfaces for data transfer? I can't reconfigure firewall..
Red hat ES3.0, nutch-2005-08-25 (>nutch-0.7).
$ java -version
java version "1.4.2-01"
Java(TM) 2 Runtime Environment, Standard Edition (build Blackdown-1.4.2-01)
Java HotSpot(TM) 64-Bit Server VM (build Blackdown-1.4.2-01, mixed mode)
Thanks for your time!
--
Best regards,
Chernodarov Egor
-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
_______________________________________________
Nutch-developers mailing list
Nutch-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nutch-developers
