Both of those are good points. I'll submit a patch. - Aaron
On Wed, Jul 22, 2009 at 6:24 PM, Ted Dunning<[email protected]> wrote: > To amplify David's point, why is the argument a Mapper.Context rather than > MapContext? > > Also, why is the Mapper.Context not static? > > On Wed, Jul 22, 2009 at 5:29 PM, David Hall <[email protected]> wrote: > >> This is nice, but doesn't it suffer from the same problem? MRUnit uses >> the mapred API, which is deprecated, and the new API doesn't use >> OutputCollector, but a non-static inner class. >> >> -- David >> >> On Wed, Jul 22, 2009 at 4:52 PM, Aaron Kimball<[email protected]> wrote: >> > Hi David, >> > >> > I wrote a contrib module called MRUnit >> > (http://issues.apache.org/jira/browse/hadoop-5518) designed to allow >> > unit tests for mappers/reducers more easily. It's slated for inclusion >> > in 0.21, not 0.20 unfortunately, but you can download the patch above >> > as well as MAPREDUCE-680 and build it against any earlier version of >> > Hadoop. Unfortunately, it doesn't currently support the new APIs >> > (e.g., with Context objects), but I imagine this could be added with >> > little difficulty. I just haven't had time to do it myself ;) If you'd >> > like to take a stab at it, I'd love some help! >> > >> > More info is at www.cloudera.com/hadoop-mrunit >> > Cheers, >> > - Aaron >> > >> > On Wed, Jul 22, 2009 at 2:49 PM, David Hall<[email protected]> wrote: >> >> Hi, >> >> >> >> I'm a student working with Apache Mahout for the Google Summer of >> >> Code. We recently moved to 0.20.0, and I was porting my code to the >> >> new API. Unfortunately, I (and the whole project team) seem to have >> >> run into a problem when it comes to testing them. >> >> >> >> Historically, we would create a Mapper in a unit test, and a special >> >> "DummyOutputCollector", which was essentially a multimap dressed up to >> >> conform to OutputCollector. In Hadoop 0.20.0, this isn't possible >> >> anymore, because Mappers take an instance of an inner class. >> >> >> >> It's of course possible to dress up the Context in something else >> >> (say, something just like an OutputCollector), and to specify that >> >> Mahout Mappers should just delegate to a method that takes an >> >> OutputCollector. But, this seems to not be very idiomatic. >> >> >> >> All this goes to say, what would be a "best practice" for testing >> >> Mappers and Reducers in 0.20.0? >> >> >> >> Thanks, >> >> David Hall >> >> >> > >> > > > > -- > Ted Dunning, CTO > DeepDyve >
