I tried that, but with 1.7GB, that will not allow me to run 1 mapper and 1
reducer concurrently (as I think when you do -Xmx1024m it tries to reserve
that physical memory?). Thus, to be safe, I set it to -Xmx768m.
The error I get when I do 1024m is this:
java.io.IOException: Cannot run program "
Darn that send button.
Anyways, so I was wondering if my understanding is correct. There will only
be the exact same number of output files as the number of reducer tasks I
set. Thus, in my output directory from the reducer, I should always see
only 18 files. However, if my understanding is cor
Maybe you need allocate larger vm- memory to use parameter -Xmx1024m
On Thu, Feb 12, 2009 at 10:56 AM, Kris Jirapinyo wrote:
> Hi all,
>I am running a data-intensive job on 18 nodes on EC2, each with just
> 1.7GB of memory. The input size is 50GB, and as a result, my mapper splits
> it up au
Hi all,
I am running a data-intensive job on 18 nodes on EC2, each with just
1.7GB of memory. The input size is 50GB, and as a result, my mapper splits
it up automatically to 786 map tasks. This runs fine. However, I am
setting the reduce task number to 18. This is where I get a java heap o
