[
https://issues.apache.org/jira/browse/CASSANDRA-5038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13514636#comment-13514636
]
Adrien Grand commented on CASSANDRA-5038:
-----------------------------------------
Hi Jake! I'm very happy that you are considering adding LZ4 support to
Cassandra!
The native impl is indeed very very fast, and the unsafe java impl is ~48%
faster at compressing data and ~1% slower at decompressing data than the Snappy
impl you use according to my last benchmarks on the calgary corpus. If you run
your own benchmarks, I'd be very interested in knowing the results.
I think the fallback to pure Java can be a plus: ideally Cassandra should first
try to load the native impl (if the JAR includes bindings for the host
architecture), then the unsafe java impl (if the JVM supports sun.misc.Unsafe)
and then the pure Java impl (which is unfortunately ~40% slower at compressing
and ~30% slower at decompressing than the unsafe impl).
I noticed in your patch that you are using the "unknown size" version of the
decompressor. It can unfortunately be slightly slower than the normal
decompressor. On large blocks, it can be faster to write the original length at
the beginning of the compressed streams to be able to use the normal
decompressor to decompress data.
I'm currently working on releasing this stuff, so hopefully this will be easier
for Cassandra to use it.
> LZ4Compressor
> -------------
>
> Key: CASSANDRA-5038
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5038
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: T Jake Luciani
> Priority: Minor
> Fix For: 1.2.1
>
> Attachments: LZ4Compressor.java, lz4-java.jar
>
>
> LZ4 is a new compression algo that's ~2x faster than Snappy.
> [~jpountz] has written a nice java port which includes a misc.Unsafe version
> that performs >= than our java snappy version.
> Details at http://blog.jpountz.net/post/28092106032/wow-lz4-is-fast
> The nice thing is this should work with java7 and be more portable.
> We can also fallback the pure java impl
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
