Both of those are good points. I'll submit a patch.
- Aaron

On Wed, Jul 22, 2009 at 6:24 PM, Ted Dunning<[email protected]> wrote:
> To amplify David's point, why is the argument a Mapper.Context rather than
> MapContext?
>
> Also, why is the Mapper.Context not static?
>
> On Wed, Jul 22, 2009 at 5:29 PM, David Hall <[email protected]> wrote:
>
>> This is nice, but doesn't it suffer from the same problem? MRUnit uses
>> the mapred API, which is deprecated, and the new API doesn't use
>> OutputCollector, but a non-static inner class.
>>
>> -- David
>>
>> On Wed, Jul 22, 2009 at 4:52 PM, Aaron Kimball<[email protected]> wrote:
>> > Hi David,
>> >
>> > I wrote a contrib module called MRUnit
>> > (http://issues.apache.org/jira/browse/hadoop-5518) designed to allow
>> > unit tests for mappers/reducers more easily. It's slated for inclusion
>> > in 0.21, not 0.20 unfortunately, but you can download the patch above
>> > as well as MAPREDUCE-680 and build it against any earlier version of
>> > Hadoop. Unfortunately, it doesn't currently support the new APIs
>> > (e.g., with Context objects), but I imagine this could be added with
>> > little difficulty. I just haven't had time to do it myself ;) If you'd
>> > like to take a stab at it, I'd love some help!
>> >
>> > More info is at www.cloudera.com/hadoop-mrunit
>> > Cheers,
>> > - Aaron
>> >
>> > On Wed, Jul 22, 2009 at 2:49 PM, David Hall<[email protected]> wrote:
>> >> Hi,
>> >>
>> >> I'm a student working with Apache Mahout for the Google Summer of
>> >> Code. We recently moved to 0.20.0, and I was porting my code to the
>> >> new API. Unfortunately, I (and the whole project team) seem to have
>> >> run into a problem when it comes to testing them.
>> >>
>> >> Historically, we would create a Mapper in a unit test, and a special
>> >> "DummyOutputCollector", which was essentially a multimap dressed up to
>> >> conform to OutputCollector. In Hadoop 0.20.0, this isn't possible
>> >> anymore, because Mappers take an instance of an inner class.
>> >>
>> >> It's of course possible to dress up the Context in something else
>> >> (say, something just like an OutputCollector), and to specify that
>> >> Mahout Mappers should just delegate to a method that takes an
>> >> OutputCollector. But, this seems to not be very idiomatic.
>> >>
>> >> All this goes to say, what would be a "best practice" for testing
>> >> Mappers and Reducers in 0.20.0?
>> >>
>> >> Thanks,
>> >> David Hall
>> >>
>> >
>>
>
>
>
> --
> Ted Dunning, CTO
> DeepDyve
>

Reply via email to