Mapper's Number depends on your inputformat.
Default Inputformat try to treat every file block of a file as a InputSplit.
And you will get the same number of mappers as the number of your
inputsplits.
try to configure "mapred.min.split.size" to reduce the number of your mapper
if you want to.
And I don't know why your reducer is just one. Anyone knows?
On Tue, Oct 7, 2008 at 9:06 AM, Andy Li <[EMAIL PROTECTED]> wrote:
> Dears,
>
> Sorry, I did not mean to cross post. But the previous article was
> accidentally posted to the HBase user list. I would like to bring it back
> to the Hadoop user since it is confusing me a lot and it is mainly
> MapReduce
> related.
>
> Currently running version hadoop-0.18.1 on 25 nodes. Map and Reduce Task
> Capacity is 92. When I do this in my MapReduce program:
>
> ============= SAMPLE CODE =============
> JobConf jconf = new JobConf(conf, TestTask.class);
> jconf.setJobName("my.test.TestTask");
> jconf.setOutputKeyClass(Text.class);
> jconf.setOutputValueClass(Text.class);
> jconf.setOutputFormat(TextOutputFormat.class);
> jconf.setMapperClass(MyMapper.class);
> jconf.setCombinerClass(MyReducer.class);
> jconf.setReducerClass(MyReducer.class);
> jconf.setInputFormat(TextInputFormat.class);
> try {
> jconf.setNumMapTasks(5);
> jconf.setNumReduceTasks(3);
> JobClient.runJob(jconf);
> } catch (Exception e) {
> e.printStackTrace();
> }
> ============= ============= =============
>
> When I run the job, I'm always getting 300 mappers and 1 reducers from the
> JobTracker webpage running on the default port 50030.
> No matter how I configure the numbers in methods "setNumMapTasks" and
> "setNumReduceTasks", I get the same result.
> Does anyone know why this is happening?
> Am I missing something or misunderstand something in the picture? =(
>
> Here's a reference to the parameters we have override in "hadoop-site.xml".
> ===============
> <property>
> <name>mapred.tasktracker.map.tasks.maximum</name>
> <value>4</value>
> </property>
>
> <property>
> <name>mapred.tasktracker.reduce.tasks.maximum</name>
> <value>4</value>
> </property>
> ================
> other parameters are default from hadoop-default.xml.
>
> Any idea how this is happening?
>
> Any inputs are appreciated.
>
> Thanks,
> -Andy
>
