[
https://issues.apache.org/jira/browse/CASSANDRA-9264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14707820#comment-14707820
]
Jonathan Shook commented on CASSANDRA-9264:
-------------------------------------------
[~aweisberg]
Am I correct in assuming that you agree with the need for the checkums, but
simply want a method that is simple to reason about as well as more inline with
the related data? Is there an opportunity here to consider this topic for
inclusion in the table format tickets?
> Cassandra should not persist files without checksums
> ----------------------------------------------------
>
> Key: CASSANDRA-9264
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9264
> Project: Cassandra
> Issue Type: Wish
> Reporter: Ariel Weisberg
> Fix For: 3.x
>
>
> Even if checksums aren't validated on the read side every time it is helpful
> to have them persisted with checksums so that if a corrupted file is
> encountered you can at least validate that the issue is corruption and not an
> application level error that generated a corrupt file.
> We should standardize on conventions for how to checksum a file and which
> checksums to use so we can ensure we get the best performance possible.
> For a small checksum I think we should use CRC32 because the hardware support
> appears quite good.
> For cases where a 4-byte checksum is not enough I think we can look at either
> xxhash64 or MurmurHash3.
> The problem with xxhash64 is that output is only 8-bytes. The problem with
> MurmurHash3 is that the Java implementation is slow. If we can live with
> 8-bytes and make it easy to switch hash implementations I think xxhash64 is a
> good choice because we already ship a good implementation with LZ4.
> I would also like to see hashes always prefixed by a type so that we can swap
> hashes without running into pain trying to figure out what hash
> implementation is present. I would also like to avoid making assumptions
> about the number of bytes in a hash field where possible keeping in mind
> compatibility and space issues.
> Hashing after compression is also desirable over hashing before compression.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
