What OS are you using with Nutch, and what version of JVM?
If its Linux, paste the output of "ulimit -a", if its BSD use "limits".
You can also try inserting "-Xms2000m" before you set the max heap, so it would
look like "-Xms2000m -Xmx2000m".
I'm also assuming you have at least 2g free of RAM, or even more?
----- Original Message ----
From: Gal Nitzan <[EMAIL PROTECTED]>
To: [email protected]
Sent: Friday, January 19, 2007 10:57:01 AM
Subject: java.lang.OutOfMemoryError - trunk
Thanks Sean,
I get out of memory errors.
I have set max heap for both nutch and hadoop 2000mb each but it doesn't
seem to affect anything. The out of memory happenes immediately after start
of a task.
Any idea?
java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:2786)
at
java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:94)
at java.io.DataOutputStream.write(DataOutputStream.java:90)
at org.apache.hadoop.io.Text.writeString(Text.java:399)
at org.apache.nutch.parse.Outlink.write(Outlink.java:52)
at org.apache.nutch.parse.ParseData.write(ParseData.java:163)
at org.apache.nutch.parse.ParseImpl.write(ParseImpl.java:55)
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:323)
at org.apache.nutch.parse.ParseSegment.map(ParseSegment.java:96)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:48)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:183)
at
org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1367)-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Nutch-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/nutch-general
