What are the exact files you are using for the mapper and reducer from the cloud9 package?
On Tue, Oct 5, 2010 at 2:15 PM, Pramy Bhats <pramybh...@googlemail.com>wrote: > Hi Ed, > > I was trying to benchmark some application code available online. > http://github.com/lintool/Cloud9 > > For the program computing concurrentmatrix strips. However, the code itself > is problematic because it throws heap-space error for even very small data > sets. > > thanks, > --Pramod > > > > On Tue, Oct 5, 2010 at 5:50 PM, ed <hadoopn...@gmail.com> wrote: > > > Hi Pramod, > > > > How much memory does each node in your cluster have? > > > > What type of processors do those nodes have? (dual core, quad core, dual > > quad core? etc..) > > > > In what step are you seeing the heap space error (mapper or reducer?) > > > > It's quite possible that you're mapper or reducer code could be improved > to > > reduce heap space usage. > > > > ~Ed > > > > On Tue, Oct 5, 2010 at 10:05 AM, Marcos Medrado Rubinelli < > > marc...@buscape-inc.com> wrote: > > > > > You can set the mapred.tasktracker.map.tasks.maximum and > > > mapred.tasktracker.reduce.tasks.maximum properties in your > > mapred-site.xml > > > file, but you may also want to check your current > mapred.child.java.opts > > and > > > mapred.child.ulimit values to make sure they aren't overriding the 4GB > > you > > > set globally. > > > > > > Cheers, > > > Marcos > > > > > > Hi, > > >> > > >> I am trying to run a job on my hadoop cluster, where I get > consistently > > >> get > > >> heap space error. > > >> > > >> I increased the heap-space to 4 GB in hadoop-env.sh and reboot the > > >> cluster. > > >> However, I still get the heap space error. > > >> > > >> > > >> One of things, I want to try is to reduce the number of map / reduce > > >> process > > >> per machine. Currently each machine can have 2 maps and 2 reduce > process > > >> running. > > >> > > >> > > >> I want to configure the hadoop to run 1 map and 1 reduce per machine > to > > >> give > > >> more heap space per process. > > >> > > >> How can I configure the number of maps and number of reducer per node > ? > > >> > > >> > > >> thanks in advance, > > >> -- Pramod > > >> > > >> > > > > > >
