The job could be designed to use one reducer
On 2011-05-20, at 7:19 AM, praveenesh kumar wrote:
> Hello everyone,
>
> I am using wordcount application to test on my hadoop cluster of 5 nodes.
> The file size is around 5 GB.
> Its taking around 2 min - 40 sec for execution.
> But when I am check
I am using the wordcount example that comes along with hadoop.
How can I configure it to make it use multiple reducers.
I guess mutiple reducers will make it run more fast .. Does it ??
On Fri, May 20, 2011 at 6:51 PM, James Seigel Tynt wrote:
> The job could be designed to use one reducer
>
>
what does your mapred-site.xml file say?
I've used wordcount and had close to 12 reduces running on a 6
datanode cluster on a 3 GB file.
I have a configuration in there which says:
mapred.reduce.tasks = 12
The reason I chose 12 was because it was recommended that I choose 2x
number of tasktrack
Hi,
My mapred-site.xml is pretty simple.
mapred.job.tracker
ub13:54311
The host and port that the MapReduce job tracker runs at. If
"local", then jobs are run in-process as a single map and reduce task.
where I should put the settings that you are saying ??
On Fri, May 20, 2011 at 6:
rijdag 20 mei 2011 15:26
> To: common-user@hadoop.apache.org
> Subject: Re: Why Only 1 Reducer is running ??
>
> what does your mapred-site.xml file say?
>
> I've used wordcount and had close to 12 reduces running on a 6
> datanode cluster on a 3 GB file.
>
>
>
) (
> http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/mapreduce/Job.html#setNumReduceTasks(int)
> ).
>
> Cheers,
> Evert
>
>
> > -Original Message-
> > From: modemide [mailto:modem...@gmail.com]
> > Sent: vrijdag 20 mei 2011 15:26
to 2).
>> * You can set the default number of reduce tasks with mapred.reduce.tasks
>> (default set to 1 - this causes your single reducer).
>> * Your job can try to override this setting by calling
>> Job.setNumReduceTasks(INT) (
>> http://hadoop.apache.org/common/docs/cu
