changing key comparator used with chain mappers

2011-03-09 Thread John Sanda
If I want to change how keys are sorted prior to the reduce, my understanding is that I can do this with JobConf.setOutputKeyComparatorClass(). I am trying to implement a job using ChainReducer such that I have, Map 1 | Reduce | Map 2 and I want to use my own comparator for Map 2 so that all keys

Re: using output from one job as input to another

2011-03-02 Thread John Sanda
n Wed, Mar 2, 2011 at 11:19 PM, Harsh J wrote: > Hello, > > On Thu, Mar 3, 2011 at 7:51 AM, John Sanda wrote: > > The output path created from the first job is a directory, and it the > file > > in that directory that has a name like part-r- that I want to feed as > &

using output from one job as input to another

2011-03-02 Thread John Sanda
Hi I am new to Hadoop, so maybe I am missing something obvious. I have written a small map reduce program that runs two jobs. I want the output of the first job to serve as the input to the second job. Here is what my driver code looks like: public int run(String[] args) throws Exception { Con