Mark, You can do it either ways. Create the connection object for the database in the configure() or setup() method of the mapper (depending on which api you are using) and insert the record from the mapper function. You dont have to have a reducer. If you create an output format, the mapper can directly write to it. In essence you'll be doing the same thing. Its easier to create an output format if you'll be writing more of such code.
-Amandeep Amandeep Khurana Computer Science Graduate Student University of California, Santa Cruz On Tue, May 11, 2010 at 7:15 PM, Mark Kerzner <markkerz...@gmail.com> wrote: > Hi, Nick, > > should I then Provide the RecordWriter implementation in the OutputFormat, > which will connect to the database and write a record to it, instead of to > HDFS? > > Thank you, > Mark > > On Tue, May 11, 2010 at 9:08 PM, Jones, Nick <nick.jo...@amd.com> wrote: > > > Hi Mark, > > It would be better to create an outputformat instead of directly > connecting > > from the mapper. The outputformat would be called regardless of the > > existence of the reducers. > > > > Make sure and set the job setNumReduceTasks(0). (I'm not sure setting the > > class to null would work.) > > > > Nick > > Sent by radiation. > > > > ----- Original Message ----- > > From: Mark Kerzner <markkerz...@gmail.com> > > To: core-u...@hadoop.apache.org <core-u...@hadoop.apache.org> > > Sent: Tue May 11 21:02:05 2010 > > Subject: Import the results into SimpleDB > > > > Hi, > > > > I want a Hadoop job that will simply take each line of the input text > file > > and store it (after parsing) in a database, like SimpleDB. > > > > Can I put this code into Mapper, make no call to "collect" in it, and > have > > no reducers at all? Do I set the reduce class to > > null, conf.setReducerClass(null)? or not set it at all? > > > > Thank you, > > Mark > > > > >
