Re: Processing 10MB files in Hadoop

Fri, 27 Nov 2009 16:08:24 -0800

Ok. I have set the number on maps to about 1760 (11 nodes * 16 cores/node * 10 as recommended by Hadoop documentation) and my job still takes several hours to run instead of one.
Can be the overhead added by Hadoop that big? I mean I have over 30000 small tasks (about one minute), each one starting its own JVM. 

























					Reply via email to