[
https://issues.apache.org/jira/browse/CASSANDRA-9096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14541828#comment-14541828
]
Branimir Lambov commented on CASSANDRA-9096:
--------------------------------------------
Patch up for review
[here|https://github.com/apache/cassandra/compare/trunk...blambov:9096-icompressor],
tests
[here|http://cassci.datastax.com/view/Dev/view/blambov/job/blambov-9096-icompressor-testall/].
Improves the {{ICompressor}} interface:
- Clarifies buffer support by changing the misleadingly named
{{useDirectOutputBuffers}} into a pair of methods supplying the preferred and
supported buffer types.
- Introduces {{BufferType}} enumeration to avoid passing around buffer type
requirements as opaque booleans.
- Changes ByteBuffer compression methods to use specified offsets and lengths
to avoid having to duplicate for thread-safety.
- Makes direct (and memory-mapped) buffers a common ground that all compressors
support by implementing a solution for {{DeflateCompressor}}.
- Removes {{WrappedByteBuffer}} and the clunkiness around it by providing an
upper bound for required compressed buffer size for {{DeflateCompressor}}.
- Shifts content a bit in {{CompressorTest}} to make sure offsets are properly
tested. Due to [a bug in
LZ4-java|https://github.com/jpountz/lz4-java/issues/64] sliced buffers (which
we don't currently use) are not included.
- Tests all buffer type combinations the compressors claim to support, not just
the preferred.
At the moment {{SnappyCompressor}} has to duplicate buffers as the version of
snappy-java we use does not give any other option. The effect of this should
not be noticeable, but if necessary this can be improved by either:
- updating to current snappy-java and using unsafe methods to compress using
buffer addresses, or
- writing a patch for snappy-java to add the required methods, which could
easily also include full support for heap buffers.
> Improve ByteBuffer compression interface
> ----------------------------------------
>
> Key: CASSANDRA-9096
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9096
> Project: Cassandra
> Issue Type: Improvement
> Reporter: Branimir Lambov
> Fix For: 3.x
>
>
> Now that we have a few uses of compression/decompression on ByteBuffers it is
> time to finalize the interface before it becomes set in stone with 3.0. The
> current code has some shortcomings:
> - The interface uses the buffers' positions and limits instead of accepting
> offset and length as parameters. This necessitates that the buffers be
> duplicated before they can be compressed for thread-safety, something that
> adds burden to the caller, is prone to being forgotten, and we could
> generally do without for performance.
> - The direct/non-direct buffer support needs to be more clearly defined. The
> current {{useDirectOutputByteBuffers}} is not named well.
> - If we don't want to support non-direct buffers everywhere as a fallback, we
> should clearly state the decision and rationale.
> - How should {{WrappedByteBuffer}} treat direct/indirect buffers?
> - More testing is necessary as e.g. errors in {{DeflateCompressor}} were only
> caught in CASSANDRA-6809.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
