[jira] [Commented] (CASSANDRA-9428) Implement hints compression
[ https://issues.apache.org/jira/browse/CASSANDRA-9428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15068541#comment-15068541 ] Joshua McKenzie commented on CASSANDRA-9428: +1, pending CI: ||branch||testall||dtest|| |[9428|https://github.com/josh-mckenzie/cassandra/tree/9428]|[testall|http://cassci.datastax.com/view/Dev/view/josh-mckenzie/job/josh-mckenzie-9428-testall]|[dtest|http://cassci.datastax.com/view/Dev/view/josh-mckenzie/job/josh-mckenzie-9428-dtest]| |[9428_trunk|https://github.com/josh-mckenzie/cassandra/tree/9428_trunk]|[testall|http://cassci.datastax.com/view/Dev/view/josh-mckenzie/job/josh-mckenzie-9428_trunk-testall]|[dtest|http://cassci.datastax.com/view/Dev/view/josh-mckenzie/job/josh-mckenzie-9428_trunk-dtest]| > Implement hints compression > --- > > Key: CASSANDRA-9428 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9428 > Project: Cassandra > Issue Type: Sub-task > Components: Coordination >Reporter: Aleksey Yeschenko >Assignee: Blake Eggleston > Fix For: 3.0.x > > > CASSANDRA-6230 is being implemented with compression in mind, but it's not > going to be implemented by the original ticket. > Adding it on top should be relatively straight-forward, and important, since > there are several users in the wild that use compression interface for > encryption purposes. DSE is one of them (but isn't the only one). Losing > encryption capabilities would be a regression. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9428) Implement hints compression
[
https://issues.apache.org/jira/browse/CASSANDRA-9428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15064411#comment-15064411
]
Joshua McKenzie commented on CASSANDRA-9428:
Feedback:
* In {{CompressedChecksummedDataInput.reBufferStandard:L65}}, consider
{{ICompressor.initialCompressedBufferLength}} rather than 5% allocation
overhead.
* {{HintsWriter.bufferWrites}}: I'm not crazy about us having a volatile
increment in the critical path that's only used in a testing context.
* In {{HintsCompressionTest.multiFlushAndDeserializeTest}}, bitshifting on init
of {{bufferSize}} is unnecessary.
* Would prefer {{HintsCompressionTest}} to exercise all available compressor
types as a matter of general hygiene.
Nits:
* extra space in yaml before new param
* extra space before closing brace in HintsWriter.create()
* unused imports in HintsCompressionTest
* some extra whitespace (doubled) in
{{HintsCompressionTest.multiFlushAndDeserializeTest}}
I'm fine with both the caveats you mention above. Looking good so far!
> Implement hints compression
> ---
>
> Key: CASSANDRA-9428
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9428
> Project: Cassandra
> Issue Type: Sub-task
> Components: Coordination
>Reporter: Aleksey Yeschenko
>Assignee: Blake Eggleston
> Fix For: 3.0.x
>
>
> CASSANDRA-6230 is being implemented with compression in mind, but it's not
> going to be implemented by the original ticket.
> Adding it on top should be relatively straight-forward, and important, since
> there are several users in the wild that use compression interface for
> encryption purposes. DSE is one of them (but isn't the only one). Losing
> encryption capabilities would be a regression.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9428) Implement hints compression
[ https://issues.apache.org/jira/browse/CASSANDRA-9428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15065084#comment-15065084 ] Blake Eggleston commented on CASSANDRA-9428: I've pushed up some additional commits to my branch addressing your points. Thanks for suggesting I exercise all compressor types, it uncovered some problems with the way I was allocating buffers, which I fixed. > Implement hints compression > --- > > Key: CASSANDRA-9428 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9428 > Project: Cassandra > Issue Type: Sub-task > Components: Coordination >Reporter: Aleksey Yeschenko >Assignee: Blake Eggleston > Fix For: 3.0.x > > > CASSANDRA-6230 is being implemented with compression in mind, but it's not > going to be implemented by the original ticket. > Adding it on top should be relatively straight-forward, and important, since > there are several users in the wild that use compression interface for > encryption purposes. DSE is one of them (but isn't the only one). Losing > encryption capabilities would be a regression. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9428) Implement hints compression
[ https://issues.apache.org/jira/browse/CASSANDRA-9428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15062888#comment-15062888 ] Blake Eggleston commented on CASSANDRA-9428: Here's my initial implementation: https://github.com/bdeggleston/cassandra/commits/9428 A few things to note: * The behavior of HintsWriter.Session.append has been altered slightly. When the size of a hint is larger than the remaining buffer, the buffer is first flushed to disk, then the hint is written into the newly cleared buffer. Previously, both the hint and the previous data were flushed to disk. I did this to avoid writing very small compressed blocks to disk or having to concatenate the two byte buffers. * When opening readers, an uncompressed reader is always instantiated first, then upgraded to a compressed reader if the file is compressed. This is because the hints descriptor containing the compressor metadata is written to the head of the file, and it needs to be written uncompressed. > Implement hints compression > --- > > Key: CASSANDRA-9428 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9428 > Project: Cassandra > Issue Type: Sub-task > Components: Coordination >Reporter: Aleksey Yeschenko >Assignee: Blake Eggleston > Fix For: 3.0.x > > > CASSANDRA-6230 is being implemented with compression in mind, but it's not > going to be implemented by the original ticket. > Adding it on top should be relatively straight-forward, and important, since > there are several users in the wild that use compression interface for > encryption purposes. DSE is one of them (but isn't the only one). Losing > encryption capabilities would be a regression. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
