Moritz,
I'm not sure what you're doing, but raising the number of mapers in your
configuration isn't a 'hint'.
The number of mapers that you can run will depend on your configuration. You
mention an i7 which is a quad core cpu, but you don't mention the amount of
memory you have available, or what else you run on the machine. You don't want
hadoop to swap.
If your initial m/r jobs are taking input from a file, the default behavior is
to create one map/reduce task per block. So if your initial input file is <
64MB and you have kept your default block size of 64MB, then you will only have
one map/reduce task.
I haven't played with Hadoop in a single node / pseudo distributed
environment... just in a distributed environment but I believe that the
functionality is the same.
HTH
-Mike
PS. Please take my advice with a grain of salt. It's 5:00am and I haven't had
my first cup of coffee yet. ;-)
> From: moritzk...@googlemail.com
> Date: Fri, 16 Jul 2010 11:03:19 +0200
> Subject: Single Node with multiple mappers?
> To: common-user@hadoop.apache.org
>
> Hi everyone,
>
> I was curious if there is any option to use Hadoop in single node mode
> in a way, that enables the process to use more system ressources.
> Right now, Hadoop uses one mapper and one reducer, leaving my i7 with
> about 20% CPU usage (1 core for Hadoop, .5 cores for my OS) basically
> idling.
> Raising the number of map tasks doesn't seem to do much, as this
> parameter seems to more of a hint anyway. Still, I have lots of CPU
> time and RAM left. Any hints on how to use them?
>
> thanks in advance,
> Moritz
_________________________________________________________________
The New Busy is not the old busy. Search, chat and e-mail from your inbox.
http://www.windowslive.com/campaign/thenewbusy?ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_3
