On Mon, Jan 18, 2010 at 1:08 PM, Alex Kozlov <[email protected]> wrote:
> You can specify the compression/codec in the file writer (is this what you > are asking?). > > > SequenceFile.createWriter(fs, conf, path, key.getClass(), value.getClass(), > SequenceFile.CompressionType.BLOCK, codec); > > You can also create your own FileOutputFormat. > > If you're using one of the built in FileOutputFormat subclasses, you can use FileOutputFormat.setOutputCompressorClass to set the codec. If you're using MultipleOutputs, you will probably have to override some functions in order to create different RecordWriters with different codecs, as Alex suggested above. -Todd > On Mon, Jan 18, 2010 at 12:53 PM, Ted Yu <[email protected]> wrote: > > > Hi, > > mapred.output.compress is set to true in hadoop-site.xml > > My question is how can I specify different compression codecs > > programmatically ? > > > > For example, normally the output is gzip compressed. But a small portion > of > > output needs to be LZO compressed. > > > > Thanks > > >
