[jira] [Updated] (HBASE-3691) Add compressor support for 'snappy', google's compressor
[ https://issues.apache.org/jira/browse/HBASE-3691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-3691: - Attachment: 3691-addendum.txt Removed snappy test; the plumbing is not usually in place (Thanks for noticing Ted) > Add compressor support for 'snappy', google's compressor > > > Key: HBASE-3691 > URL: https://issues.apache.org/jira/browse/HBASE-3691 > Project: HBase > Issue Type: Task >Reporter: stack >Priority: Critical > Fix For: 0.90.7, 0.92.0 > > Attachments: 3691-addendum.txt, hbase-snappy-0.90.6.patch, > hbase-snappy-3691-trunk-002.patch, hbase-snappy-3691-trunk-003.patch, > hbase-snappy-3691-trunk-004.patch, hbase-snappy-3691-trunk.patch > > > http://code.google.com/p/snappy/ is apache licensed. > bq. Snappy is a compression/decompression library. It does not aim for > maximum compression, or compatibility with any other compression library; > instead, it aims for very high speeds and reasonable compression. For > instance, compared to the fastest mode of zlib, Snappy is an order of > magnitude faster for most inputs, but the resulting compressed files are > anywhere from 20% to 100% bigger. On a single core of a Core i7 processor in > 64-bit mode, Snappy compresses at about 250 MB/sec or more and decompresses > at about 500 MB/sec or more. > bq. Snappy is widely used inside Google, in everything from BigTable and > MapReduce to our internal RPC systems. (Snappy has previously been referred > to as "Zippy" in some presentations and the likes.) > Lets get it in. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3691) Add compressor support for 'snappy', google's compressor
[ https://issues.apache.org/jira/browse/HBASE-3691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-3691: - Fix Version/s: 0.90.7 Applied to 0.90 branch. > Add compressor support for 'snappy', google's compressor > > > Key: HBASE-3691 > URL: https://issues.apache.org/jira/browse/HBASE-3691 > Project: HBase > Issue Type: Task >Reporter: stack >Priority: Critical > Fix For: 0.90.7, 0.92.0 > > Attachments: hbase-snappy-0.90.6.patch, > hbase-snappy-3691-trunk-002.patch, hbase-snappy-3691-trunk-003.patch, > hbase-snappy-3691-trunk-004.patch, hbase-snappy-3691-trunk.patch > > > http://code.google.com/p/snappy/ is apache licensed. > bq. Snappy is a compression/decompression library. It does not aim for > maximum compression, or compatibility with any other compression library; > instead, it aims for very high speeds and reasonable compression. For > instance, compared to the fastest mode of zlib, Snappy is an order of > magnitude faster for most inputs, but the resulting compressed files are > anywhere from 20% to 100% bigger. On a single core of a Core i7 processor in > 64-bit mode, Snappy compresses at about 250 MB/sec or more and decompresses > at about 500 MB/sec or more. > bq. Snappy is widely used inside Google, in everything from BigTable and > MapReduce to our internal RPC systems. (Snappy has previously been referred > to as "Zippy" in some presentations and the likes.) > Lets get it in. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3691) Add compressor support for 'snappy', google's compressor
[ https://issues.apache.org/jira/browse/HBASE-3691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Waterson updated HBASE-3691: -- Attachment: hbase-snappy-0.90.6.patch Yes, I have. I've applied hbase-snappy-0.90.5.patch and it seems to be working on a small (but heavily loaded) HBase 0.90.6 cluster. > Add compressor support for 'snappy', google's compressor > > > Key: HBASE-3691 > URL: https://issues.apache.org/jira/browse/HBASE-3691 > Project: HBase > Issue Type: Task >Reporter: stack >Priority: Critical > Fix For: 0.92.0 > > Attachments: hbase-snappy-0.90.6.patch, > hbase-snappy-3691-trunk-002.patch, hbase-snappy-3691-trunk-003.patch, > hbase-snappy-3691-trunk-004.patch, hbase-snappy-3691-trunk.patch > > > http://code.google.com/p/snappy/ is apache licensed. > bq. Snappy is a compression/decompression library. It does not aim for > maximum compression, or compatibility with any other compression library; > instead, it aims for very high speeds and reasonable compression. For > instance, compared to the fastest mode of zlib, Snappy is an order of > magnitude faster for most inputs, but the resulting compressed files are > anywhere from 20% to 100% bigger. On a single core of a Core i7 processor in > 64-bit mode, Snappy compresses at about 250 MB/sec or more and decompresses > at about 500 MB/sec or more. > bq. Snappy is widely used inside Google, in everything from BigTable and > MapReduce to our internal RPC systems. (Snappy has previously been referred > to as "Zippy" in some presentations and the likes.) > Lets get it in. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3691) Add compressor support for 'snappy', google's compressor
[ https://issues.apache.org/jira/browse/HBASE-3691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-3691: - Resolution: Fixed Hadoop Flags: [Reviewed] Status: Resolved (was: Patch Available) Committed to TRUNK. Thanks Nicolas and Nichole for the patches. Nicolas, I added your howto above to the book into the compression appendix. Thanks. > Add compressor support for 'snappy', google's compressor > > > Key: HBASE-3691 > URL: https://issues.apache.org/jira/browse/HBASE-3691 > Project: HBase > Issue Type: Task >Reporter: stack >Priority: Critical > Fix For: 0.92.0 > > Attachments: hbase-snappy-3691-trunk-002.patch, > hbase-snappy-3691-trunk-003.patch, hbase-snappy-3691-trunk-004.patch, > hbase-snappy-3691-trunk.patch > > > http://code.google.com/p/snappy/ is apache licensed. > bq. Snappy is a compression/decompression library. It does not aim for > maximum compression, or compatibility with any other compression library; > instead, it aims for very high speeds and reasonable compression. For > instance, compared to the fastest mode of zlib, Snappy is an order of > magnitude faster for most inputs, but the resulting compressed files are > anywhere from 20% to 100% bigger. On a single core of a Core i7 processor in > 64-bit mode, Snappy compresses at about 250 MB/sec or more and decompresses > at about 500 MB/sec or more. > bq. Snappy is widely used inside Google, in everything from BigTable and > MapReduce to our internal RPC systems. (Snappy has previously been referred > to as "Zippy" in some presentations and the likes.) > Lets get it in. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3691) Add compressor support for 'snappy', google's compressor
[ https://issues.apache.org/jira/browse/HBASE-3691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Telford updated HBASE-3691: Attachment: hbase-snappy-3691-trunk-004.patch Moved Compression.Algorithm.SNAPPY to end of enum to retain backwards compatibility with existing HFiles. Otherwise, patch is same as 003 > Add compressor support for 'snappy', google's compressor > > > Key: HBASE-3691 > URL: https://issues.apache.org/jira/browse/HBASE-3691 > Project: HBase > Issue Type: Task >Reporter: stack >Priority: Critical > Fix For: 0.92.0 > > Attachments: hbase-snappy-3691-trunk-002.patch, > hbase-snappy-3691-trunk-003.patch, hbase-snappy-3691-trunk-004.patch, > hbase-snappy-3691-trunk.patch > > > http://code.google.com/p/snappy/ is apache licensed. > bq. Snappy is a compression/decompression library. It does not aim for > maximum compression, or compatibility with any other compression library; > instead, it aims for very high speeds and reasonable compression. For > instance, compared to the fastest mode of zlib, Snappy is an order of > magnitude faster for most inputs, but the resulting compressed files are > anywhere from 20% to 100% bigger. On a single core of a Core i7 processor in > 64-bit mode, Snappy compresses at about 250 MB/sec or more and decompresses > at about 500 MB/sec or more. > bq. Snappy is widely used inside Google, in everything from BigTable and > MapReduce to our internal RPC systems. (Snappy has previously been referred > to as "Zippy" in some presentations and the likes.) > Lets get it in. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3691) Add compressor support for 'snappy', google's compressor
[ https://issues.apache.org/jira/browse/HBASE-3691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nichole Treadway updated HBASE-3691: Attachment: (was: hbase-snappy-3691-trunk-003.patch) > Add compressor support for 'snappy', google's compressor > > > Key: HBASE-3691 > URL: https://issues.apache.org/jira/browse/HBASE-3691 > Project: HBase > Issue Type: Task >Reporter: stack >Priority: Critical > Fix For: 0.92.0 > > Attachments: hbase-snappy-3691-trunk-002.patch, > hbase-snappy-3691-trunk.patch > > > http://code.google.com/p/snappy/ is apache licensed. > bq. Snappy is a compression/decompression library. It does not aim for > maximum compression, or compatibility with any other compression library; > instead, it aims for very high speeds and reasonable compression. For > instance, compared to the fastest mode of zlib, Snappy is an order of > magnitude faster for most inputs, but the resulting compressed files are > anywhere from 20% to 100% bigger. On a single core of a Core i7 processor in > 64-bit mode, Snappy compresses at about 250 MB/sec or more and decompresses > at about 500 MB/sec or more. > bq. Snappy is widely used inside Google, in everything from BigTable and > MapReduce to our internal RPC systems. (Snappy has previously been referred > to as "Zippy" in some presentations and the likes.) > Lets get it in. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3691) Add compressor support for 'snappy', google's compressor
[ https://issues.apache.org/jira/browse/HBASE-3691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nichole Treadway updated HBASE-3691: Attachment: hbase-snappy-3691-trunk-003.patch Accidentally selected wrong license option. > Add compressor support for 'snappy', google's compressor > > > Key: HBASE-3691 > URL: https://issues.apache.org/jira/browse/HBASE-3691 > Project: HBase > Issue Type: Task >Reporter: stack >Priority: Critical > Fix For: 0.92.0 > > Attachments: hbase-snappy-3691-trunk-002.patch, > hbase-snappy-3691-trunk-003.patch, hbase-snappy-3691-trunk.patch > > > http://code.google.com/p/snappy/ is apache licensed. > bq. Snappy is a compression/decompression library. It does not aim for > maximum compression, or compatibility with any other compression library; > instead, it aims for very high speeds and reasonable compression. For > instance, compared to the fastest mode of zlib, Snappy is an order of > magnitude faster for most inputs, but the resulting compressed files are > anywhere from 20% to 100% bigger. On a single core of a Core i7 processor in > 64-bit mode, Snappy compresses at about 250 MB/sec or more and decompresses > at about 500 MB/sec or more. > bq. Snappy is widely used inside Google, in everything from BigTable and > MapReduce to our internal RPC systems. (Snappy has previously been referred > to as "Zippy" in some presentations and the likes.) > Lets get it in. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3691) Add compressor support for 'snappy', google's compressor
[ https://issues.apache.org/jira/browse/HBASE-3691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nichole Treadway updated HBASE-3691: Attachment: hbase-snappy-3691-trunk-003.patch Thanks for the patch...I made a few additional changes in HColumnDescriptor, and I updated the test files to include snappy. I noticed there are places in the hbase.avro classes where snappy support would need to be added in. Is it ok to add these changes in the patch, or do the avro classes need to be auto-generated somehow? > Add compressor support for 'snappy', google's compressor > > > Key: HBASE-3691 > URL: https://issues.apache.org/jira/browse/HBASE-3691 > Project: HBase > Issue Type: Task >Reporter: stack >Priority: Critical > Fix For: 0.92.0 > > Attachments: hbase-snappy-3691-trunk-002.patch, > hbase-snappy-3691-trunk.patch > > > http://code.google.com/p/snappy/ is apache licensed. > bq. Snappy is a compression/decompression library. It does not aim for > maximum compression, or compatibility with any other compression library; > instead, it aims for very high speeds and reasonable compression. For > instance, compared to the fastest mode of zlib, Snappy is an order of > magnitude faster for most inputs, but the resulting compressed files are > anywhere from 20% to 100% bigger. On a single core of a Core i7 processor in > 64-bit mode, Snappy compresses at about 250 MB/sec or more and decompresses > at about 500 MB/sec or more. > bq. Snappy is widely used inside Google, in everything from BigTable and > MapReduce to our internal RPC systems. (Snappy has previously been referred > to as "Zippy" in some presentations and the likes.) > Lets get it in. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3691) Add compressor support for 'snappy', google's compressor
[ https://issues.apache.org/jira/browse/HBASE-3691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Telford updated HBASE-3691: Attachment: hbase-snappy-3691-trunk-002.patch Seems I'd accidentally based the patch against 0.90.2, not trunk. Re-based against trunk. > Add compressor support for 'snappy', google's compressor > > > Key: HBASE-3691 > URL: https://issues.apache.org/jira/browse/HBASE-3691 > Project: HBase > Issue Type: Task >Reporter: stack >Priority: Critical > Fix For: 0.92.0 > > Attachments: hbase-snappy-3691-trunk-002.patch, > hbase-snappy-3691-trunk.patch > > > http://code.google.com/p/snappy/ is apache licensed. > bq. Snappy is a compression/decompression library. It does not aim for > maximum compression, or compatibility with any other compression library; > instead, it aims for very high speeds and reasonable compression. For > instance, compared to the fastest mode of zlib, Snappy is an order of > magnitude faster for most inputs, but the resulting compressed files are > anywhere from 20% to 100% bigger. On a single core of a Core i7 processor in > 64-bit mode, Snappy compresses at about 250 MB/sec or more and decompresses > at about 500 MB/sec or more. > bq. Snappy is widely used inside Google, in everything from BigTable and > MapReduce to our internal RPC systems. (Snappy has previously been referred > to as "Zippy" in some presentations and the likes.) > Lets get it in. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3691) Add compressor support for 'snappy', google's compressor
[ https://issues.apache.org/jira/browse/HBASE-3691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Telford updated HBASE-3691: Attachment: hbase-snappy-3691-trunk.patch The patch itself. > Add compressor support for 'snappy', google's compressor > > > Key: HBASE-3691 > URL: https://issues.apache.org/jira/browse/HBASE-3691 > Project: HBase > Issue Type: Task >Reporter: stack >Priority: Critical > Fix For: 0.92.0 > > Attachments: hbase-snappy-3691-trunk.patch > > > http://code.google.com/p/snappy/ is apache licensed. > bq. Snappy is a compression/decompression library. It does not aim for > maximum compression, or compatibility with any other compression library; > instead, it aims for very high speeds and reasonable compression. For > instance, compared to the fastest mode of zlib, Snappy is an order of > magnitude faster for most inputs, but the resulting compressed files are > anywhere from 20% to 100% bigger. On a single core of a Core i7 processor in > 64-bit mode, Snappy compresses at about 250 MB/sec or more and decompresses > at about 500 MB/sec or more. > bq. Snappy is widely used inside Google, in everything from BigTable and > MapReduce to our internal RPC systems. (Snappy has previously been referred > to as "Zippy" in some presentations and the likes.) > Lets get it in. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3691) Add compressor support for 'snappy', google's compressor
[ https://issues.apache.org/jira/browse/HBASE-3691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Telford updated HBASE-3691: Release Note: Added support for Google's Snappy compression codec. Status: Patch Available (was: Open) As far as I can tell this is all that's required in Hbase to add support for Snappy. Since it's an optional runtime dependency, and we can guarantee the class name (SnappyCodec) and it's interface (CompressionCodec) we're not actually blocked by the addition of the CompressionCodec itself (HADOOP-7206). I've tested this against the preliminary support for Snappy in HADOOP-7206, as far as I can tell they're simply waiting on some licensing constraints to be resolved (which doesn't affect this patch). > Add compressor support for 'snappy', google's compressor > > > Key: HBASE-3691 > URL: https://issues.apache.org/jira/browse/HBASE-3691 > Project: HBase > Issue Type: Task >Reporter: stack >Priority: Critical > Fix For: 0.92.0 > > Attachments: hbase-snappy-3691-trunk.patch > > > http://code.google.com/p/snappy/ is apache licensed. > bq. Snappy is a compression/decompression library. It does not aim for > maximum compression, or compatibility with any other compression library; > instead, it aims for very high speeds and reasonable compression. For > instance, compared to the fastest mode of zlib, Snappy is an order of > magnitude faster for most inputs, but the resulting compressed files are > anywhere from 20% to 100% bigger. On a single core of a Core i7 processor in > 64-bit mode, Snappy compresses at about 250 MB/sec or more and decompresses > at about 500 MB/sec or more. > bq. Snappy is widely used inside Google, in everything from BigTable and > MapReduce to our internal RPC systems. (Snappy has previously been referred > to as "Zippy" in some presentations and the likes.) > Lets get it in. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
