[
https://issues.apache.org/jira/browse/HBASE-2404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12852870#action_12852870
]
Jonathan Gray commented on HBASE-2404:
--------------------------------------
bq. the way HFile uses LZO has functional equivalence to prefix compression on
keys
What exactly do you mean by that? I understand that repeated prefixes will
compress well with lots of codecs, but is there something special about how we
use LZO in HFile that makes it more prefix friendly? And if we added prefix
compression, would that then mean we wouldn't use LZO? As I understand it,
those things can still be complimentary, and from what I recall BigTable uses
both prefix compression and LZW.
> native fast compression codec
> -----------------------------
>
> Key: HBASE-2404
> URL: https://issues.apache.org/jira/browse/HBASE-2404
> Project: Hadoop HBase
> Issue Type: New Feature
> Reporter: Andrew Purtell
>
> We often recommend enabling LZO on tables, most users see big wins. LZO is
> roughly comparable to BigTable LZW, also the way HFile uses LZO has
> functional equivalence to prefix compression on keys. However, LZO is GPL
> licensed, so a series of install steps are required:
> http://wiki.apache.org/hadoop/UsingLzoCompression . It's easy to miss a step
> or get it wrong. If so, all writes on a table (re)configured to use LZO will
> fail.
> Hadoop, well, Java, has native support for gzip compression but it is too
> slow generally; is a good option however for archival tables.
> This issue is about considering bundling or creating a comparable alternate
> to LZO which is ASF 2.0 license compatible.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
