koo created CASSANDRA-21685:
-------------------------------
Summary: Size calculation mismatch in EncodingStats and
SerializationHeader local deletion time
Key: CASSANDRA-21685
URL: https://issues.apache.org/jira/browse/CASSANDRA-21685
Project: Apache Cassandra
Issue Type: Bug
Reporter: koo
While working on CASSANDRA-21535, CompactionsCQLTest failed
(testCompactionInvalidRowDeletion and testIndexedReaderRowDeletion). The cause
is that some serializers calculate the size in a different way from how they
write the value.
h3. Problem
In EncodingStats.Serializer, the write path casts the value to int, but the
size path keeps it as long:
{code:java}
// serialize()
out.writeUnsignedVInt32((int)(stats.minLocalDeletionTime -
DELETION_TIME_EPOCH));
// serializedSize()
TypeSizes.sizeofUnsignedVInt(stats.minLocalDeletionTime - DELETION_TIME_EPOCH)
{code}
SerializationHeader has the same pattern:
{code:java}
// writeLocalDeletionTime()
out.writeUnsignedVInt32((int) (localDeletionTime - stats.minLocalDeletionTime));
// localDeletionTimeSerializedSize()
TypeSizes.sizeofUnsignedVInt(localDeletionTime - stats.minLocalDeletionTime);
{code}
If the value fits in a signed int, both give the same result. But when
minLocalDeletionTime holds an invalid or capped deletion time, the delta is
around 2.85e9 and does not fit in a signed int:
* size path: sizeofUnsignedVInt(2852087294L) -> 5 bytes
* write path: the (int) cast wraps to -1442880002, and writeUnsignedVInt32
sign-extends it -> 9 bytes
So serializedSize() returns a smaller value than the bytes actually written.
h3. Proposed fix
Apply the same (int) cast in the size calculation, in both
EncodingStats.Serializer.serializedSize() and
SerializationHeader.localDeletionTimeSerializedSize(). This makes the size
match the written bytes. It does not change any serialized byte, so there is no
compatibility impact.
This change does not fix the root problem described below. But I think it is
still clearly useful on its own, because the calculated size should always
match the actual bytes written. This is why I am opening this ticket separately.
h3. Out of scope
The written value itself does not round-trip. deserialize() reads it back with
readUnsignedVInt32(), so the original value is not restored
(minLocalDeletionTime comes back as -2).
Fixing this needs a change to the serialized format, which is used by both
SSTables and messaging. So we would also need to handle compatibility with
older versions and with data that is already written. I think the people who
maintain the project over the long term should decide the policy for this, so I
am leaving it out of this ticket.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
