I'm curious if you have been able to track down the cause of the error?
We've seen similar problems with loading data and I've discovered if I
presort my data before the load that things go a LOT smoother.
When running queries against our data sometimes we've seen it where the
jobtracker
I am still trying to see how to narrow this down. Is it possible to set
heapdumponoutofmemoryerror option on these individual tasks?
On Mon, Mar 5, 2012 at 5:49 PM, Mohit Anchlia mohitanch...@gmail.comwrote:
Sorry for multiple emails. I did find:
2012-03-05 17:26:35,636 INFO
All I see in the logs is:
2012-03-05 17:26:36,889 FATAL org.apache.hadoop.mapred.TaskTracker: Task:
attempt_201203051722_0001_m_30_1 - Killed : Java heap space
Looks like task tracker is killing the tasks. Not sure why. I increased
heap from 512 to 1G and still it fails.
On Mon, Mar 5,
Sorry for multiple emails. I did find:
2012-03-05 17:26:35,636 INFO
org.apache.pig.impl.util.SpillableMemoryManager: first memory handler call-
Usage threshold init = 715849728(699072K) used = 575921696(562423K)
committed = 715849728(699072K) max = 715849728(699072K)
2012-03-05 17:26:35,719
I am trying to run PFPGrowth but I keep receiving this Java heap space
error at the end of the first step/beginning of second step.
I am using the following parameters: -method mapreduce -regex [\\t]
-s 5 -g 55000
Output:
..
10/11/11 08:12:56 INFO mapred.JobClient: map 100% reduce
Dear All,
I am running a hadoop job processing data. The output of map is really
large, and it spill 15 times. So I was trying to set io.sort.mb = 256
instead of 100. And I leave everything else default. I am using 0.20.2
version. And when I run the job, I got the following errors:
2010-03-11
Moving to mapreduce-user@, bcc: common-user
Have you tried bumping up the heap for the map task?
Since you are setting io.sort.mb to 256M, pls set heap-size to 512M at
least, if not more.
mapred.child.java.opts - -Xmx512M or -Xmx1024m
Arun
On Mar 11, 2010, at 8:24 AM, Boyu Zhang wrote:
Hi Arun,
I did what you said, and it seems to work. Thanks a lot! I guess I do not
completely understand how the tuning parameters affect each other. Thanks!
Boyu
On Thu, Mar 11, 2010 at 12:27 PM, Arun C Murthy a...@yahoo-inc.com wrote:
Moving to mapreduce-user@, bcc: common-user
Have you
