I created
https://issues.apache.org/jira/browse/HADOOP-5750
to follow this up.
Thanks
Jothi
On 4/27/09 10:10 PM, "Jothi Padmanabhan" <joth...@yahoo-inc.com> wrote:
> Ryan,
>
> I observed this behavior too -- Partitioner does not seems to work with the
> new API exactly for the reason you have mentioned. Till this gets fixed, you
> probably need to use the old API.
>
> Jothi
>
>
> On 4/27/09 7:14 PM, "Ryan Farris" <farri...@gmail.com> wrote:
>
>> Is there some magic to get a Partitioner working on .20.0? Setting
>> the partitioner class on the Job object doesn't take, hadoop always
>> uses the HashPartitioner. Looking through the source code, it looks
>> like the MapOutputBuffer in MapTask only ever fetches the
>> "mapred.partitioner.class", and doesn't check for new api's
>> "mapreduce.partitioner.class", but I'm not confident in my
>> understanding of how things work.
>>
>> I was eventually able to get my test program working correctly by:
>> 1) Creating a partitioner that extends the deprecated
>> org.apache.hadoop.mapred.Partitioner class.
>> 2) Calling job.getConfiguration().set("mapred.partitioner.class",
>> DeprecatedTestPartitioner.class.getCanonicalName());
>> 3) Commenting out line 395 of org.apache.hadoop.mapreduce.Job.java,
>> where it asserts that "mapred.partitioner.class" is null
>>
>> But I'm assuming editing the hadoop core sourcecode is not the
>> intended path. Am I missing some simple switch or something?
>>
>> rf
>
