[
https://issues.apache.org/jira/browse/HDFS-2115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13210351#comment-13210351
]
Roy Roye commented on HDFS-2115:
--------------------------------
This http://www.eecs.berkeley.edu/Pubs/TechRpts/2010/EECS-2010-36.pdf says:
We analyzed how compression can improve performance and energy efficiency for
MapReduce workloads. Our results show that compression provides 35-60% energy
savings for read heavy jobs as well as jobs with highly compressible data.
Based on our measurements, we construct an algorithm that examines per-job data
characteristics and IO patterns, and decides when and where to use compression.
> Transparent compression in HDFS
> -------------------------------
>
> Key: HDFS-2115
> URL: https://issues.apache.org/jira/browse/HDFS-2115
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: data-node, hdfs client
> Reporter: Todd Lipcon
>
> In practice, we find that a lot of users store text data in HDFS without
> using any compression codec. Improving usability of compressible formats like
> Avro/RCFile helps with this, but we could also help many users by providing
> an option to transparently compress data as it is stored.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
