Hi,
Here's the output from when it freezes. Sorry it's a bit verbose,
wasn't sure what we're looking for so I've included it all:
051230 131519 fetching
http://www.municipalaffairs.gov.ab.ca/fco/pdf/ab-clan6-1.pdf
Full thread dump Java HotSpot(TM) Client VM (1.4.2_10-b03 mixed mode):
"fetcher9" prio=1 tid=0x082f5b40 nid=0x1442 waiting for monitor entry
[5097a000..5097a228]
at
org.apache.nutch.fetcher.Fetcher$FetcherThread.outputPage(Fetcher.java:277)
- waiting to lock <0x590efa58> (a
org.apache.nutch.io.ArrayFile$Writer)
at
org.apache.nutch.fetcher.Fetcher$FetcherThread.handleFetch(Fetcher.java:261)
at
org.apache.nutch.fetcher.Fetcher$FetcherThread.run(Fetcher.java:148)
"fetcher8" prio=1 tid=0x08165ed8 nid=0x1442 waiting for monitor entry
[509fb000..509fb228]
at
org.apache.nutch.fetcher.Fetcher$FetcherThread.outputPage(Fetcher.java:277)
- waiting to lock <0x590efa58> (a
org.apache.nutch.io.ArrayFile$Writer)
at
org.apache.nutch.fetcher.Fetcher$FetcherThread.handleFetch(Fetcher.java:261)
at
org.apache.nutch.fetcher.Fetcher$FetcherThread.run(Fetcher.java:148)
"fetcher7" prio=1 tid=0x0816bb28 nid=0x1442 waiting for monitor entry
[50a7c000..50a7c228]
at
org.apache.nutch.fetcher.Fetcher$FetcherThread.outputPage(Fetcher.java:277)
- waiting to lock <0x590efa58> (a
org.apache.nutch.io.ArrayFile$Writer)
at
org.apache.nutch.fetcher.Fetcher$FetcherThread.handleFetch(Fetcher.java:261)
at
org.apache.nutch.fetcher.Fetcher$FetcherThread.run(Fetcher.java:148)
"fetcher6" prio=1 tid=0x0816b9c0 nid=0x1442 waiting for monitor entry
[50afd000..50afd228]
at
org.apache.nutch.fetcher.Fetcher$FetcherThread.outputPage(Fetcher.java:277)
- waiting to lock <0x590efa58> (a
org.apache.nutch.io.ArrayFile$Writer)
at
org.apache.nutch.fetcher.Fetcher$FetcherThread.handleFetch(Fetcher.java:261)
at
org.apache.nutch.fetcher.Fetcher$FetcherThread.run(Fetcher.java:148)
"fetcher5" prio=1 tid=0x081672f0 nid=0x1442 waiting for monitor entry
[50b7e000..50b7e228]
at
org.apache.nutch.fetcher.Fetcher$FetcherThread.outputPage(Fetcher.java:277)
- waiting to lock <0x590efa58> (a
org.apache.nutch.io.ArrayFile$Writer)
at
org.apache.nutch.fetcher.Fetcher$FetcherThread.handleFetch(Fetcher.java:261)
at
org.apache.nutch.fetcher.Fetcher$FetcherThread.run(Fetcher.java:148)
"fetcher4" prio=1 tid=0x082f73a0 nid=0x1442 waiting for monitor entry
[50bff000..50bff228]
at
org.apache.nutch.fetcher.Fetcher$FetcherThread.outputPage(Fetcher.java:277)
- waiting to lock <0x590efa58> (a
org.apache.nutch.io.ArrayFile$Writer)
at
org.apache.nutch.fetcher.Fetcher$FetcherThread.handleFetch(Fetcher.java:261)
at
org.apache.nutch.fetcher.Fetcher$FetcherThread.run(Fetcher.java:148)
"fetcher3" prio=1 tid=0x08168768 nid=0x1442 waiting for monitor entry
[50c80000..50c80228]
at
org.apache.nutch.fetcher.Fetcher$FetcherThread.outputPage(Fetcher.java:277)
- waiting to lock <0x590efa58> (a
org.apache.nutch.io.ArrayFile$Writer)
at
org.apache.nutch.fetcher.Fetcher$FetcherThread.handleFetch(Fetcher.java:261)
at
org.apache.nutch.fetcher.Fetcher$FetcherThread.run(Fetcher.java:148)
"fetcher2" prio=1 tid=0x08168320 nid=0x1442 waiting for monitor entry
[50d01000..50d01228]
at
org.apache.nutch.fetcher.Fetcher$FetcherThread.outputPage(Fetcher.java:277)
- waiting to lock <0x590efa58> (a
org.apache.nutch.io.ArrayFile$Writer)
at
org.apache.nutch.fetcher.Fetcher$FetcherThread.handleFetch(Fetcher.java:261)
at
org.apache.nutch.fetcher.Fetcher$FetcherThread.run(Fetcher.java:148)
"fetcher1" prio=1 tid=0x081640c0 nid=0x1442 runnable [50d86000..50d87228]
at java.util.zip.Deflater.deflateBytes(Native Method)
at java.util.zip.Deflater.deflate(Deflater.java:287)
- locked <0x61da0728> (a java.util.zip.Deflater)
at
java.util.zip.DeflaterOutputStream.deflate(DeflaterOutputStream.java:154)
at
java.util.zip.DeflaterOutputStream.write(DeflaterOutputStream.java:114)
at java.util.zip.GZIPOutputStream.write(GZIPOutputStream.java:72)
- locked <0x61da0960> (a java.util.zip.GZIPOutputStream)
at
org.apache.nutch.io.WritableUtils.writeCompressedByteArray(WritableUtils.java:53)
at org.apache.nutch.protocol.Content.write(Content.java:81)
at
org.apache.nutch.io.SequenceFile$Writer.append(SequenceFile.java:137)
at org.apache.nutch.io.MapFile$Writer.append(MapFile.java:127)
- locked <0x590efa90> (a org.apache.nutch.io.ArrayFile$Writer)
at org.apache.nutch.io.ArrayFile$Writer.append(ArrayFile.java:39)
- locked <0x590efa90> (a org.apache.nutch.io.ArrayFile$Writer)
at
org.apache.nutch.fetcher.Fetcher$FetcherThread.outputPage(Fetcher.java:278)
- locked <0x590efa58> (a org.apache.nutch.io.ArrayFile$Writer)
at
org.apache.nutch.fetcher.Fetcher$FetcherThread.handleFetch(Fetcher.java:261)
at
org.apache.nutch.fetcher.Fetcher$FetcherThread.run(Fetcher.java:148)
"fetcher0" prio=1 tid=0x08163d68 nid=0x1442 waiting for monitor entry
[50e08000..50e08228]
at
org.apache.nutch.fetcher.Fetcher$FetcherThread.outputPage(Fetcher.java:277)
- waiting to lock <0x590efa58> (a
org.apache.nutch.io.ArrayFile$Writer)
at
org.apache.nutch.fetcher.Fetcher$FetcherThread.handleFetch(Fetcher.java:261)
at
org.apache.nutch.fetcher.Fetcher$FetcherThread.run(Fetcher.java:148)
"Signal Dispatcher" daemon prio=1 tid=0x080974d0 nid=0x1442 runnable [0..0]
"Finalizer" daemon prio=1 tid=0x08092448 nid=0x1442 in Object.wait()
[51588000..51588228]
at java.lang.Object.wait(Native Method)
- waiting on <0x58fe8c40> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:111)
- locked <0x58fe8c40> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:127)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)
"Reference Handler" daemon prio=1 tid=0x080918a0 nid=0x1442 in
Object.wait() [51609000..51609228]
at java.lang.Object.wait(Native Method)
- waiting on <0x58fe8ca8> (a java.lang.ref.Reference$Lock)
at java.lang.Object.wait(Object.java:429)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:115)
- locked <0x58fe8ca8> (a java.lang.ref.Reference$Lock)
"main" prio=1 tid=0x0805be28 nid=0x1442 waiting on condition
[bfffd000..bfffd2dc]
at java.lang.Thread.sleep(Native Method)
at org.apache.nutch.fetcher.Fetcher.run(Fetcher.java:351)
at org.apache.nutch.fetcher.Fetcher.main(Fetcher.java:488)
"VM Thread" prio=1 tid=0x08090638 nid=0x1442 runnable
"VM Periodic Task Thread" prio=1 tid=0x08099cd8 nid=0x1442 waiting on
condition
"Suspend Checker Thread" prio=1 tid=0x08096b28 nid=0x1442 runnable
Thanks again.
Andrzej Bialecki wrote:
Insurance Squared Inc. wrote:
Hi All,
We're experiencing problems with nutch freezing sporadically when
fetching. Not really to sure where to even start investigating.
Some digging into the archives suggested memory issues, so we did the
following:
TOMCAT_OPTS=" -Xmx1024M" to increase Tomcat memory and
NUTCH_HEAPSIZE=1500 to increase Nutch memory.
No effect - it's still freezing. Running on a current version of
mandrake linux on it's own box, 2 gigs of ram, 2.4ghz P4. I believe
we're running version .7 of nutch Does anyone have any suggestions as
to where I should start investigating?
Do you use the parse-pdf plugin? Please do a thread dump of the stuck
process (Ctrl-E, if I'm not mistaken).
-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems? Stop! Download the new AJAX search engine that makes
searching your log files as easy as surfing the web. DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
Nutch-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/nutch-general
