Hi,
2010/11/25 Shai Erera <ser...@gmail.com>:
> Is there a way to make MapReduce create exactly N Mappers? More
> specifically, if say my data can be split to 200 Mappers, and I have only
> 100 cores, how can I ensure only 100 Mappers will be created? The number of
> cores is not something I know in advance, so writing a special InputFormat
> might be tricky, unless I can query Hadoop for the available # of cores (in
> the entire cluster).
You can configure on a node by node basis how many map and reduce
tasks can be started by the task tracker on that node.
This is done via the conf/mapred-site.xml using these two settings:
mapred.tasktracker.{map|reduce}.tasks.maximum
Have a look at this page for more information
http://hadoop.apache.org/common/docs/current/cluster_setup.html
--
Met vriendelijke groeten,
Niels Basjes
