Add ability to split text files compressed with lzo
---------------------------------------------------
Key: HADOOP-4640
URL: https://issues.apache.org/jira/browse/HADOOP-4640
Project: Hadoop Core
Issue Type: Improvement
Components: io, mapred
Reporter: Johan Oskarsson
Priority: Trivial
Right now any file compressed with lzop will be processed by one mapper. This
is a shame since the lzo algorithm would be very suitable for large log files
and similar common hadoop data sets. The compression rate is not the best out
there but the decompression speed is amazing. Since lzo writes compressed data
in blocks it would be possible to make an input format that can split the files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
