>>
>>> Can¹t seem to get past this heap size error, any ideas where to look? Below
>>> are my heap size settings, at least the ones I attempted to increase.
>>>
>>> Thanks in advance for any thoughts.
>>>
>>>
>>> # The maximum amount of heap to use, in MB. Default is 1000.
>>> export HADOOP_HEAPSIZE=10000
>>>
>>>
>>>
>>> <name>mapred.child.java.opts</name>
>>> <value>-Xmx1048576m</value>
>>> </property>
>>>
>>>
>>>
>>>
>>>
>>> 11/11/12 17:38:27 INFO hpc.Driver: Jar Name:
>>> /home/hadoop/querry/HadoopTest.jar
>>> 0 [main] INFO nccs.hpc.Driver - Jar Name:
>>> /home/hadoop/querry/HadoopTest.jar
>>> 0 [main] INFO nccs.hpc.Driver - Jar Name:
>>> /home/hadoop/querry/HadoopTest.jar
>>> 11/11/12 17:38:27 WARN mapred.JobClient: Use GenericOptionsParser for
>>> parsing the arguments. Applications should implement Tool for the same.
>>> 146 [main] WARN org.apache.hadoop.mapred.JobClient - Use
>>> GenericOptionsParser for parsing the arguments. Applications should
>>> implement Tool for the same.
>>> 146 [main] WARN org.apache.hadoop.mapred.JobClient - Use
>>> GenericOptionsParser for parsing the arguments. Applications should
>>> implement Tool for the same.
>>> 11/11/12 17:38:30 INFO input.FileInputFormat: Total input paths to process :
>>> 2329
>>> 3330 [main] INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat -
>>> Total input paths to process : 2329
>>> 3330 [main] INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat -
>>> Total input paths to process : 2329
>>> 11/11/12 17:38:34 INFO mapred.JobClient: Running job: job_201111121210_0001
>>> 7919 [main] INFO org.apache.hadoop.mapred.JobClient - Running job:
>>> job_201111121210_0001
>>> 7919 [main] INFO org.apache.hadoop.mapred.JobClient - Running job:
>>> job_201111121210_0001
>>> 11/11/12 17:38:35 INFO mapred.JobClient: map 0% reduce 0%
>>> 8922 [main] INFO org.apache.hadoop.mapred.JobClient - map 0% reduce 0%
>>> 8922 [main] INFO org.apache.hadoop.mapred.JobClient - map 0% reduce 0%
>>> 11/11/12 17:40:37 INFO mapred.JobClient: map 1% reduce 0%
>>> 130058 [main] INFO org.apache.hadoop.mapred.JobClient - map 1% reduce 0%
>>> 130058 [main] INFO org.apache.hadoop.mapred.JobClient - map 1% reduce 0%
>>> 11/11/12 17:42:17 INFO mapred.JobClient: map 2% reduce 0%
>>> 230162 [main] INFO org.apache.hadoop.mapred.JobClient - map 2% reduce 0%
>>> 230162 [main] INFO org.apache.hadoop.mapred.JobClient - map 2% reduce 0%
>>> 11/11/12 17:43:47 INFO mapred.JobClient: map 3% reduce 0%
>>> 320251 [main] INFO org.apache.hadoop.mapred.JobClient - map 3% reduce 0%
>>> 320251 [main] INFO org.apache.hadoop.mapred.JobClient - map 3% reduce 0%
>>> 11/11/12 17:45:26 INFO mapred.JobClient: map 4% reduce 0%
>>> 419346 [main] INFO org.apache.hadoop.mapred.JobClient - map 4% reduce 0%
>>> 419346 [main] INFO org.apache.hadoop.mapred.JobClient - map 4% reduce 0%
>>> 11/11/12 17:46:59 INFO mapred.JobClient: map 5% reduce 0%
>>> 512435 [main] INFO org.apache.hadoop.mapred.JobClient - map 5% reduce 0%
>>> 512435 [main] INFO org.apache.hadoop.mapred.JobClient - map 5% reduce 0%
>>> 11/11/12 17:48:26 INFO mapred.JobClient: Task Id :
>>> attempt_201111121210_0001_r_000000_0, Status : FAILED
>>> 599520 [main] INFO org.apache.hadoop.mapred.JobClient - Task Id :
>>> attempt_201111121210_0001_r_000000_0, Status : FAILED
>>> 599520 [main] INFO org.apache.hadoop.mapred.JobClient - Task Id :
>>> attempt_201111121210_0001_r_000000_0, Status : FAILED
>>> Error: java.lang.OutOfMemoryError: Java heap space
>>> at
>>> org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.shuffleInMe
>>> mory(ReduceTask.java:1685)
>>> at
>>> org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.getMapOutpu
>>> t(ReduceTask.java:1545)
>>> at
>>> org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.copyOutput(
>>> ReduceTask.java:1394)
>>> at
>>> org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.run(ReduceT
>>> ask.java:1326)
>>>
