[
https://issues.apache.org/jira/browse/CASSANDRA-21751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-21751:
-----------------------------------------
Bug Category: Parent values: Correctness(12982)Level 1 values: Recoverable
Corruption / Loss(12986)
Complexity: Normal
Component/s: Legacy/Local Write-Read Paths
Discovered By: User Report
Fix Version/s: 5.0.x
6.x
7.x
Severity: Normal
Status: Open (was: Triage Needed)
> Decimal validation accepts a 4-byte value (a scale with no unscaled value),
> which Cassandra itself cannot read, compare, flush or compact
> -----------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-21751
> URL: https://issues.apache.org/jira/browse/CASSANDRA-21751
> Project: Apache Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
> Reporter: Nadav Har'El
> Priority: Normal
> Fix For: 5.0.x, 6.x, 7.x
>
>
> A {{decimal}} value is a 4-byte {{int}} scale followed by a {{varint}}
> unscaled value. {{DecimalSerializer.validate()}} only rejects non-empty
> values shorter than 4 bytes, so it accepts a 4-byte value consisting of the
> scale alone. Nothing in Cassandra can interpret such a value:
> {{DecimalSerializer.deserialize()}} passes the empty remainder to {{{}new
> BigInteger(byte[]){}}}, which throws {{{}NumberFormatException: Zero length
> BigInteger{}}}. As a result, one such value, inserted with
> {{blobAsDecimal()}} or as a bound value, causes server errors on reads and
> writes. As a clustering key it also makes memtable flush and compaction of
> the table fail.
> Tested on 5.0.9 and 6.0-alpha2, with identical results.
> h3. Reproducer
> {code:sql}
> CREATE KEYSPACE ks WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};
> CREATE TABLE ks.t (p int, c decimal, v decimal, PRIMARY KEY (p, c));
> -- Accepted (also accepted as a 4-byte bound value):
> INSERT INTO ks.t (p, c, v) VALUES (1, 1, blobAsDecimal(0x00000000));
> SELECT decimalAsBlob(v) FROM ks.t WHERE p = 1; -- 0x00000000, stored as is
> -- Every operation that interprets the value fails with
> -- ServerError: java.lang.NumberFormatException: Zero length BigInteger
> SELECT CAST(v AS text) FROM ks.t WHERE p = 1;
> SELECT JSON p, c, v FROM ks.t WHERE p = 1;
> UPDATE ks.t SET v = 2 WHERE p = 1 AND c = 1 IF v = 0;
> -- As a clustering key:
> INSERT INTO ks.t (p, c) VALUES (2, blobAsDecimal(0x00000000)); -- accepted
> INSERT INTO ks.t (p, c) VALUES (2, 1); --
> WriteFailure
> SELECT decimalAsBlob(c) FROM ks.t WHERE p = 2 AND c = 0; -- ReadFailure
> {code}
> Then {{nodetool flush ks t}} fails with "error: Zero length BigInteger". The
> same failure happens with arithmetic ({{{}v + 1{}}}) and {{{}sum(v){}}}.
> h3. Impact
> # *Server errors on reads:* {{{}CAST{}}}, {{{}SELECT JSON{}}}, arithmetic,
> {{sum()}} and LWT conditions on the column all fail with a server error
> rather than a clean client error.
> # *The partition stops accepting writes:* once a 4-byte clustering key is in
> a partition, inserting any other clustering key into that partition fails
> (WriteFailure). The memtable insert has to compare the new key with the
> 4-byte one, and that comparison throws. Reads that compare against the key
> (e.g. {{WHERE c = 0}} or {{{}c > -1{}}}) fail with ReadFailure.
> # *Flush fails:* {{MetadataCollector.updateClusteringValues()}} compares
> every row's clustering key with the min/max so far. So a memtable holding a
> 4-byte clustering key plus any other row of the table, even in another
> partition, cannot be flushed:
> {noformat}
> ERROR [PerDiskMemtableFlushWriter_0:2] JVMStabilityInspector -- Exception in
> thread Thread[PerDiskMemtableFlushWriter_0:2,...]
> java.lang.NumberFormatException: Zero length BigInteger
> at java.base/java.math.BigInteger.<init>(Unknown Source)
> at
> org.apache.cassandra.serializers.DecimalSerializer.deserialize(DecimalSerializer.java:38)
> at
> org.apache.cassandra.db.marshal.AbstractType.compose(AbstractType.java:133)
> at
> org.apache.cassandra.db.marshal.AbstractType.compareComposed(AbstractType.java:115)
> at
> org.apache.cassandra.db.marshal.DecimalType.compareCustom(DecimalType.java:88)
> at
> org.apache.cassandra.db.marshal.AbstractType.compare(AbstractType.java:214)
> at
> org.apache.cassandra.db.ClusteringComparator.compareComponent(ClusteringComparator.java:191)
> at
> org.apache.cassandra.db.ClusteringComparator.compare(ClusteringComparator.java:148)
> at
> org.apache.cassandra.io.sstable.metadata.MetadataCollector.updateClusteringValues(MetadataCollector.java:311)
> at
> org.apache.cassandra.io.sstable.format.SortedTableWriter.addRow(SortedTableWriter.java:213)
> ...
> {noformat}
> After the failed flush, the next {{nodetool flush}} of the table reports
> success without writing anything, and the rows of the failed memtable are
> still served from memory. When the table was later dropped, Cassandra logged
> "Failed to force-recycle all segments; at least one segment is still in use
> with dirty CFs." I did not test what commit log replay does after a restart.
> # *Compaction fails:* if the 4-byte key was flushed alone into its own
> sstable, compacting that sstable with any other sstable of the table fails
> with the same exception, from {{CompactionTask.runMayThrow()}} through
> {{{}MetadataCollector.updateClusteringValues(){}}}.
> # *Clients cannot read it:* the Java driver 3.12.1 bundled with Cassandra
> throws {{NumberFormatException: Zero length BigInteger}} from
> {{{}TypeCodec.decimal().deserialize(){}}}. The Python driver fails to decode
> the result column ("invalid literal for int() with base 16") and then marks
> the whole connection defunct. ScyllaDB's fork of gocql (v1.18.3) explicitly
> rejects it ("the length of the data should be 0 or more than 5").
> h3. Why this is a validation gap, not a legitimate encoding of zero
> One might argue that a 4-byte decimal is a valid encoding of zero, and that
> {{deserialize()}} should be fixed to accept it instead. It should not:
> # *The protocol spec defines zero as one byte.* Section 5.6 of
> native_protocol_v5.spec defines a decimal as a scale "followed by a varint
> encoding (see section 5.24) of the unscaled value". Section 5.24's example
> table encodes 0 as {{{}0x00{}}}. No varint is encoded as zero bytes.
> # *Cassandra's own varint type treats empty bytes as "empty", not as 0.*
> {{IntegerSerializer.deserialize()}} returns {{null}} for an empty value, and
> {{IntegerType}} sorts the empty value before all numbers. On 5.0.9 and
> 6.0-alpha2, a {{varint}} column set to {{blobAsVarint(0x)}} reads back as
> null, {{CAST(i AS text)}} and {{i + 1}} return null, and {{SELECT JSON}}
> returns an empty string (see below). Reading a decimal's empty unscaled
> varint as 0 would contradict how the same varint encoding is read everywhere
> else in Cassandra.
> {code:java}
> SELECT JSON i FROM ks.v WHERE p = 1; -- {"i": ""}
> {code}
> # *No writer produces it.* {{DecimalSerializer.serialize()}} writes
> {{{}BigInteger.toByteArray(){}}}, which is never empty (zero is
> {{{}0x00{}}}), and drivers do the same. The only ways to create a 4-byte
> decimal are {{blobAsDecimal()}} or a hand-crafted bound value.
> # *The 4-byte lower bound was never a format decision.* Until 2013,
> {{validate()}} did nothing ("no useful check for invalid decimals"). The
> current check was added during the CASSANDRA-5744 cleanup with the comment
> "We at least store the scale", i.e. as a minimal sanity check. Nothing
> indicates that a value with no unscaled part was considered and allowed.
> # *Nothing can depend on it.* Every code path that interprets such a value
> already fails, both in Cassandra and in the drivers. So rejecting it at write
> time cannot break any working application. It only turns a later server
> error, or a flush or compaction failure, into an immediate
> {{{}InvalidRequest{}}}.
> h3. Suggested fix
> In {{{}DecimalSerializer.validate(){}}}, reject non-empty values shorter than
> 5 bytes (4-byte scale plus at least one byte of unscaled value), e.g.
> "Expected 0 or at least 5 bytes".
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
