Block size and input size should not matter for the Reducer. You do have to
explicitly say the number of workers.
It defaults to 1. You do set it with just these methods. Make sure you are
setting on the right object and before you run. Look for other things that
may be overriding it.
I don't know this job maybe it is forcing 1 for some reason.
On Aug 30, 2012 9:58 AM, "Paritosh Ranjan" <pran...@xebia.com> wrote:
> If the problem is only the number of reduce tasks, then you can try to
> reduce the dfs block size. This might help in triggering multiple reducers.
> Also check the size of the mapper's output, if its greater than the block
> size ( or the mapper output is scattered in multiple files ) , then only
> multiple reducers would be triggered.
>
> HTH,
> Paritosh
>
> On 30-08-2012 12:08, C.V.Krishnakumar Iyer wrote:
>
>> Hi,
>>
>> I've already tried setting it in the code using job.setNumReduceTasks()
>> and conf.set("mapred.reduce.tasks"**,"100").
>> However, it does not seem to take the number of reducers at all, even for
>> the job that does parallel counting. Any advice would be appreciated.
>> Regards,
>> Krishnakumar.
>> On Aug 29, 2012, at 11:28 PM, 戴清灏 <rogerda...@gmail.com> wrote:
>>
>> I doubt that you specify the config in hadoop config xml file.
>>>
>>> --
>>> Regards,
>>> Q
>>>
>>>
>>>
>>> 2012/8/30 C.V. Krishnakumar Iyer <cvkrishnaku...@me.com>
>>>
>>> Hi,
>>>>
>>>> Quick question regarding PFPGrowth in Mahout 0.6:
>>>>
>>>> I see that there are no options to set the number of reducers in the
>>>> parallel counting phase of PFP Growth. It is just simple word count - so
>>>> I'm guessing it should be parallelized. But for some reason it is not!
>>>>
>>>> Is that intentional?
>>>>
>>>> Regards,
>>>> Krishnakumar.
>>>>
>>>>
>
>
