mapred.child.java.opts is not set. Should it be? ________________________________________ From: Uma Maheswara Rao G [mahesw...@huawei.com] Sent: Tuesday, February 07, 2012 2:27 AM To: common-user@hadoop.apache.org Subject: RE: out of memory running examples
What is the Java heap space you configured? for the property mapred.child.java.opts ________________________________________ From: Tim Broberg [tim.brob...@exar.com] Sent: Tuesday, February 07, 2012 3:20 PM To: common-user@hadoop.apache.org Subject: out of memory running examples I'm trying to run the basic example from hadoop/hadoop-1.0.0/docs/single_node_setup.html. I'm getting java.lang.OutOfMemoryError's when I run the grep example from that page. Stackoverflow suggests various tweaks to the command line, mapred-site.xml, or hadoop-env.sh, none of which seem to be helping in my case. When I tweak hadoop-env.sh to echo text to a file, that file doesn't show up, which suggests that hadoop-env.sh isn't even getting executed. Any hints on debugging this? - Tim. [tbroberg@san-mothra hadoop-1.0.0]$ bin/hadoop jar hadoop-examples-1.0.0.jar grep input output 'dfs[a-z.]+' Warning: $HADOOP_HOME is deprecated. 12/02/07 01:39:35 INFO util.NativeCodeLoader: Loaded the native-hadoop library 12/02/07 01:39:35 INFO mapred.FileInputFormat: Total input paths to process : 7 12/02/07 01:39:35 INFO mapred.JobClient: Running job: job_local_0001 12/02/07 01:39:35 INFO util.ProcessTree: setsid exited with exit code 0 12/02/07 01:39:35 INFO mapred.Task: Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@4c349471<mailto:org.apache.hadoop.util.LinuxResourceCalculatorPlugin@4c349471> 12/02/07 01:39:35 INFO mapred.MapTask: numReduceTasks: 1 12/02/07 01:39:35 INFO mapred.MapTask: io.sort.mb = 100 12/02/07 01:39:35 WARN mapred.LocalJobRunner: job_local_0001 java.lang.OutOfMemoryError: Java heap space at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:949) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:428) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372) at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212) 12/02/07 01:39:36 INFO mapred.JobClient: map 0% reduce 0% 12/02/07 01:39:36 INFO mapred.JobClient: Job complete: job_local_0001 12/02/07 01:39:36 INFO mapred.JobClient: Counters: 0 12/02/07 01:39:36 INFO mapred.JobClient: Job Failed: NA java.io.IOException: Job failed! at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1265) at org.apache.hadoop.examples.Grep.run(Grep.java:69) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) at org.apache.hadoop.examples.Grep.main(Grep.java:93) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68) at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139) at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:64) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:156) [tbroberg@san-mothra hadoop-1.0.0]$ ________________________________ The information and any attached documents contained in this message may be confidential and/or legally privileged. The message is intended solely for the addressee(s). If you are not the intended recipient, you are hereby notified that any use, dissemination, or reproduction is strictly prohibited and may be unlawful. If you are not the intended recipient, please contact the sender immediately by return e-mail and destroy all copies of the original message. The information and any attached documents contained in this message may be confidential and/or legally privileged. The message is intended solely for the addressee(s). If you are not the intended recipient, you are hereby notified that any use, dissemination, or reproduction is strictly prohibited and may be unlawful. If you are not the intended recipient, please contact the sender immediately by return e-mail and destroy all copies of the original message.