[jira] [Commented] (CASSANDRA-13304) Add checksumming to the native protocol
[
https://issues.apache.org/jira/browse/CASSANDRA-13304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16593106#comment-16593106
]
Jordan West commented on CASSANDRA-13304:
-
[~beobal] comments below. The protocol comments may be changes that are too
late to make at this stage but since I'm new to the ticket I wanted to leave
those observations as well.
Protocol Comments:
* It would be nice to not have to pay the cost of storing compressed lengths
when there is no compression enabled but checksums are
* It would be an improvement to allow the mixing of compressed and
uncompressed blocks in a frame. Then blocks that don’t compress well (are
larger than their input or are too small for compression to be worth it) can be
accounted for.
Other comments:
* cassandra.yaml entry for compression/checksum block size is missing
Minor nits:
* There was some conversation above about using {{ProtocolException}} instead
of {{IOException}} when the checksums don't match. It seemed like there was
agreement on using {{ProtocolException}} but the code still uses
{{IOException}}.
* Would be nice to move {{ChecksummingTransformer#readUnsignedShort}} to
something like {{ByteBufUtil#readUnsignedShort}}. Similar to
{{ByteBufferUtil#readShortLength}}.
* StartupMessage#getChecksumType/getCompressor(): I'm not sure there is much
benefit to using optional here given how its used at the call sites.
* The comment about why the frame.compress package defines an ICompressor-Like
interface was removed but is helpful since its not obvious at first. It should
probably be expanded on a bit as well.
[https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/transport/FrameCompressor.java#L37]
* The case where a connection w/ compression enabled sends an uncompressed
frame is not covered by ChecksummingTransformerTest
* The created by comment at the top of ChecksummingCompressorTest should be
removed
{{ChecksummingTransformerTest}} is also a great candidate for a simple property
based test. Since we don’t have a library in-tree currently, I’ll write
something on the side.
> Add checksumming to the native protocol
> ---
>
> Key: CASSANDRA-13304
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13304
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
>Reporter: Michael Kjellman
>Assignee: Sam Tunnicliffe
>Priority: Blocker
> Labels: client-impacting
> Fix For: 4.x
>
> Attachments: 13304_v1.diff, boxplot-read-throughput.png,
> boxplot-write-throughput.png
>
>
> The native binary transport implementation doesn't include checksums. This
> makes it highly susceptible to silently inserting corrupted data either due
> to hardware issues causing bit flips on the sender/client side, C*/receiver
> side, or network in between.
> Attaching an implementation that makes checksum'ing mandatory (assuming both
> client and server know about a protocol version that supports checksums) --
> and also adds checksumming to clients that request compression.
> The serialized format looks something like this:
> {noformat}
> * 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 3 3
> * 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
> * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> * | Number of Compressed Chunks | Compressed Length (e1)/
> * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> * / Compressed Length cont. (e1) |Uncompressed Length (e1) /
> * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> * | Uncompressed Length cont. (e1)| CRC32 Checksum of Lengths (e1)|
> * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> * | Checksum of Lengths cont. (e1)|Compressed Bytes (e1)+//
> * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> * | CRC32 Checksum (e1) ||
> * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> * |Compressed Length (e2) |
> * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> * | Uncompressed Length (e2)|
> * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> * |CRC32 Checksum of Lengths (e2) |
> * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> * | Compressed Bytes (e2) +//
> * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> * | CRC32 Checksum (e2) ||
> * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> * |Compressed
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16592916#comment-16592916 ] Patrick Bannister commented on CASSANDRA-10190: --- Quick update on this ticket. The porting work is mostly done - I had basically all the tests passing under Python 2.7 and Python 3.5 on CentOS and Ubuntu environments. Making the dtests Python 2/3 cross compatible was trivial, mostly I just had to add explicit inheritance from the object class to a few of the basic classes like dtest.py. There will be a small patch to ccm related to Unicode encoding when calling cqlsh. I've learned a bit about our test infrastructure, so I have a better idea of what will be involved for the testing environment subtask (CASSANDRA-14491). I'll continue work on this ticket after we complete CASSANDRA-14298 - that task is building momentum again so I think it will be soon. > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement >Reporter: Andrew Pennebaker >Priority: Major > Labels: cqlsh > Attachments: coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
