[ https://issues.apache.org/jira/browse/HADOOP-5793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13687655#comment-13687655 ]
Harsh J commented on HADOOP-5793: --------------------------------- Should this JIRA be moved to HBase, as it seems most useful there? > High speed compression algorithm like BMDiff > -------------------------------------------- > > Key: HADOOP-5793 > URL: https://issues.apache.org/jira/browse/HADOOP-5793 > Project: Hadoop Common > Issue Type: New Feature > Reporter: elhoim gibor > Assignee: Michele Catasta > Priority: Minor > > Add a high speed compression algorithm like BMDiff. > It gives speeds ~100MB/s for writes and ~1000MB/s for reads, compressing > 2.1billions web pages from 45.1TB in 4.2TB > Reference: > http://norfolk.cs.washington.edu/htbin-post/unrestricted/colloq/details.cgi?id=437 > 2005 Jeff Dean talk about google architecture - around 46:00. > http://feedblog.org/2008/10/12/google-bigtable-compression-zippy-and-bmdiff/ > http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=755678 > A reference implementation exists in HyperTable. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira