[
https://issues.apache.org/jira/browse/HADOOP-2402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Douglas updated HADOOP-2402:
----------------------------------
Attachment: 2402-0.patch
Inserted a 64k BufferedOutputStream to TextOutputFormat::getRecordWriter.
Effects compression similar to block, lzo compressed SequenceFile (~20%). For
comparison, lzop (command line compression utility backed by lzo lib) uses 256k
blocks (and a different file format) and compresses the same 100M sample by 60%.
As an aside, with this applied, Zip compressed text files are written
approximately 10% faster; uncompressed text about 4% faster.
> Lzo compression compresses each write from TextOutputFormat
> -----------------------------------------------------------
>
> Key: HADOOP-2402
> URL: https://issues.apache.org/jira/browse/HADOOP-2402
> Project: Hadoop
> Issue Type: Bug
> Components: io, mapred, native
> Reporter: Chris Douglas
> Fix For: 0.16.0
>
> Attachments: 2402-0.patch
>
>
> Outputting with TextOutputFormat and Lzo compression generates a file such
> that each key, tab delimiter, and value are compressed separately.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
