You can set the mapred.tasktracker.map.tasks.maximum and
mapred.tasktracker.reduce.tasks.maximum properties in your
mapred-site.xml file, but you may also want to check your current
mapred.child.java.opts and mapred.child.ulimit values to make sure they
aren't overriding the 4GB you set globally.
Cheers,
Marcos
Hi,
I am trying to run a job on my hadoop cluster, where I get consistently get
heap space error.
I increased the heap-space to 4 GB in hadoop-env.sh and reboot the cluster.
However, I still get the heap space error.
One of things, I want to try is to reduce the number of map / reduce process
per machine. Currently each machine can have 2 maps and 2 reduce process
running.
I want to configure the hadoop to run 1 map and 1 reduce per machine to give
more heap space per process.
How can I configure the number of maps and number of reducer per node ?
thanks in advance,
-- Pramod
