Re: Reflective instantiation of Mappers and Reducers

2010-04-04 Thread Tom White
Have a look at org.apache.hadoop.io.Stringifier (and DefaultStringifier), which may be helpful too. Cheers, Tom On Fri, Apr 2, 2010 at 3:35 PM, Kris Nuttycombe wrote: > Or heck... I could just base-64 encode the serialized byte arrays and > pass them as strings in the configuration. If it's goin

Re: Reflective instantiation of Mappers and Reducers

2010-04-02 Thread Kris Nuttycombe
Or heck... I could just base-64 encode the serialized byte arrays and pass them as strings in the configuration. If it's going to be a hack, might as well go all the way. On Fri, Apr 2, 2010 at 4:10 PM, Kris Nuttycombe wrote: > On Fri, Apr 2, 2010 at 3:10 PM, Owen O'Malley wrote: >> >> On Apr 2,

Re: Reflective instantiation of Mappers and Reducers

2010-04-02 Thread Kris Nuttycombe
On Fri, Apr 2, 2010 at 3:10 PM, Owen O'Malley wrote: > > On Apr 2, 2010, at 12:05 PM, Kris Nuttycombe wrote: > >> What I'm wondering is, is there any way to simply serialize a Mapper >> or Reducer object, and have the serialized instance copied, passed >> around and used everywhere instead of alwa

Re: Reflective instantiation of Mappers and Reducers

2010-04-02 Thread Owen O'Malley
On Apr 2, 2010, at 12:05 PM, Kris Nuttycombe wrote: What I'm wondering is, is there any way to simply serialize a Mapper or Reducer object, and have the serialized instance copied, passed around and used everywhere instead of always having the Mapper and Reducer instantiated by reflection? This

Reflective instantiation of Mappers and Reducers

2010-04-02 Thread Kris Nuttycombe
Hi, all, I'm new to Hadoop, and I'm finding myself having a hard time creating highly configurable Mapper and Reducer instances, due to the fact that Hadoop seems to require that Mapper and Reducer instances be instantiated through reflection, so I'm sort of wondering whether I'm doing things wro