Hi all,
I upgraded from Nutch 0.8.1 to 0.9 and thought the hung thread issues are
fixed in this version (NUTCH-344), but I am still seeing the same problem.
I am only running 10 threads for a focused crawl, and have the default
fetcher.max.crawl.delay of 30. Every time it hangs, it is followed by a NPE
- any clue? Thanks!
2007-06-19 14:40:37,670 WARN fetcher.Fetcher - Aborting with 3 hung
threads.
2007-06-19 15:01:46,518 FATAL fetcher.Fetcher -
java.lang.NullPointerException
2007-06-19 15:01:46,545 FATAL fetcher.Fetcher - at
org.apache.hadoop.fs.FSDataInputStream$Buffer.getPos(FSDataInputStream.java
:87)
2007-06-19 15:01:46,546 FATAL fetcher.Fetcher - at
org.apache.hadoop.fs.FSDataInputStream.getPos(FSDataInputStream.java:125)
2007-06-19 15:01:46,546 FATAL fetcher.Fetcher - at
org.apache.hadoop.io.SequenceFile$Reader.getPosition(SequenceFile.java:1736)
2007-06-19 15:01:46,546 FATAL fetcher.Fetcher - at
org.apache.hadoop.mapred.SequenceFileRecordReader.getProgress(
SequenceFileRecordReader.java:108)
2007-06-19 15:01:46,546 FATAL fetcher.Fetcher - at
org.apache.hadoop.mapred.MapTask$1.getProgress(MapTask.java:165)
2007-06-19 15:01:46,546 FATAL fetcher.Fetcher - at
org.apache.hadoop.mapred.MapTask$1.next(MapTask.java:155)
2007-06-19 15:01:46,546 FATAL fetcher.Fetcher - at
org.apache.nutch.fetcher.Fetcher$FetcherThread.run(Fetcher.java:115)
2007-06-19 15:01:46,546 FATAL fetcher.Fetcher - fetcher caught:
java.lang.NullPointerException
-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
Nutch-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/nutch-general
