since the whole data is 5TB. the Identity reducer still cost a lot of time.
On Mon, Jul 21, 2008 at 5:09 PM, Christian Ulrik Søttrup <[EMAIL PROTECTED]> wrote: > Hi, > > you can simply use the built in reducer that just copies the map output: > > conf.setReducerClass(org.apache.hadoop.mapred.lib.IdentityReducer.class); > > Cheers, > Christian > > > Zhou, Yunqing wrote: > >> I only use it to do something in parallel,but the reduce step will cost me >> additional several days, is it possible to make hadoop do not use a reduce >> step? >> >> Thanks >> >> >> > >
