[
https://issues.apache.org/jira/browse/HADOOP-5793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12876390#action_12876390
]
Todd Lipcon commented on HADOOP-5793:
-------------------------------------
Luke: did you implement this cleanroom in Hypertable? Do you hold the entire
copyright thereof? If so, would you consider relicensing the code as Apache 2
license for use in Hadoop?
It would be great to have something like this that is Apache 2.0 licensed so we
can freely distribute it with the rest of Hadoop, rather than making people
separately compile and install a dependency.
> High speed compression algorithm like BMDiff
> --------------------------------------------
>
> Key: HADOOP-5793
> URL: https://issues.apache.org/jira/browse/HADOOP-5793
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: elhoim gibor
> Assignee: Michele Catasta
> Priority: Minor
>
> Add a high speed compression algorithm like BMDiff.
> It gives speeds ~100MB/s for writes and ~1000MB/s for reads, compressing
> 2.1billions web pages from 45.1TB in 4.2TB
> Reference:
> http://norfolk.cs.washington.edu/htbin-post/unrestricted/colloq/details.cgi?id=437
> 2005 Jeff Dean talk about google architecture - around 46:00.
> http://feedblog.org/2008/10/12/google-bigtable-compression-zippy-and-bmdiff/
> http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=755678
> A reference implementation exists in HyperTable.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
