To get rid of empty *part files while using MultipleOutputs in the new API, LazyOutputFormat class' static method should be used to set the output format.
Details are here at the official Java docs for MultipleOutputs : http://hadoop.apache.org/docs/r2.3.0/api/org/apache/hadoop/mapreduce/lib/output/MultipleOutputs.html Regards, Shahab On Tue, Jul 1, 2014 at 9:08 PM, M. Dale <medal...@yahoo.com> wrote: > NullOutputFormat does not generate any output. Good for jobs where > counters or some other I/O are your output (for example, > http://stackoverflow.com/questions/12707726/run-a-hadoop-job-without-output-file > ). > > From Tom White's book it sounds like > org.apache.hadoop.mapreduce.lib.output.LazyOutputFormat only generates a > file if at least one key/value pair is written. > > Hope this helps, > Markus > > > On 07/01/2014 07:33 AM, Chris MacKenzie wrote: > > Hi, > > What is the anticipated usage of the above with the new api ? Is there > another way to remove the empty part-r files > > When using it with MultipleOutputs to remove empty part-r files I have no > output ;O) > > > > Regards, > > Chris MacKenzie<http://www.chrismackenziephotography.co.uk/> > <http://www.chrismackenziephotography.co.uk/> > > >
