[ https://issues.apache.org/jira/browse/HDFS-2115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13210351#comment-13210351 ]
Roy Roye commented on HDFS-2115: -------------------------------- This http://www.eecs.berkeley.edu/Pubs/TechRpts/2010/EECS-2010-36.pdf says: We analyzed how compression can improve performance and energy efficiency for MapReduce workloads. Our results show that compression provides 35-60% energy savings for read heavy jobs as well as jobs with highly compressible data. Based on our measurements, we construct an algorithm that examines per-job data characteristics and IO patterns, and decides when and where to use compression. > Transparent compression in HDFS > ------------------------------- > > Key: HDFS-2115 > URL: https://issues.apache.org/jira/browse/HDFS-2115 > Project: Hadoop HDFS > Issue Type: New Feature > Components: data-node, hdfs client > Reporter: Todd Lipcon > > In practice, we find that a lot of users store text data in HDFS without > using any compression codec. Improving usability of compressible formats like > Avro/RCFile helps with this, but we could also help many users by providing > an option to transparently compress data as it is stored. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira