[
https://issues.apache.org/jira/browse/CASSANDRA-19623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Klay updated CASSANDRA-19623:
-----------------------------
Attachment: (was: system.log)
> Read fail with "illegal RT bounds sequence" after migrating data from 2.2.19
> to 3.0.30
> --------------------------------------------------------------------------------------
>
> Key: CASSANDRA-19623
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19623
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
> Reporter: Klay
> Priority: Normal
> Fix For: 3.0.x
>
> Attachments: cassandra.yaml, data.tar.gz
>
>
> After migrating data from 2.2.19 to 3.0.30, reading the legacy data in 3.0.30
> would fail with the following exception.
> {code:java}
> INFO [main] 2024-05-08 00:42:42,397 CassandraDaemon.java:653 - Startup
> complete
> ERROR [SharedPool-Worker-2] 2024-05-08 00:42:49,975
> AbstractLocalAwareExecutorService.java:166 - Uncaught exception on thread
> Thread[SharedPool-Worker-2,10,main]
> java.lang.RuntimeException: java.lang.IllegalStateException: SSTABLE
> UnfilteredRowIterator for ks.tb has an illegal RT bounds sequence: unexpected
> end bound or boundary Marker INCL_START_BOUND(1)@1715128933051779/1715128933
> at
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2656)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:162)
> at
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:134)
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:109)
> at java.lang.Thread.run(Thread.java:750)
> Caused by: java.lang.IllegalStateException: SSTABLE UnfilteredRowIterator for
> ks.tb has an illegal RT bounds sequence: unexpected end bound or boundary
> Marker INCL_START_BOUND(1)@1715128933051779/1715128933
> at
> org.apache.cassandra.db.transform.RTBoundValidator$RowsTransformation.ise(RTBoundValidator.java:120)
> at
> org.apache.cassandra.db.transform.RTBoundValidator$RowsTransformation.applyToMarker(RTBoundValidator.java:87)
> at
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:144)
> at
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:129)
> at
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:131)
> at
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:87)
> at
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:77)
> at
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:297)
> at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.build(ReadResponse.java:187)
> at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.<init>(ReadResponse.java:180)
> at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.<init>(ReadResponse.java:176)
> at
> org.apache.cassandra.db.ReadResponse.createDataResponse(ReadResponse.java:76)
> at
> org.apache.cassandra.db.ReadCommand.createResponse(ReadCommand.java:347)
> at
> org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:1914)
> at
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2652)
> ... 5 common frames omitted {code}
> h1. Reproduce
> 1. Start up cassandra-2.2.19, single node. Set column_index_size_in_kb in
> cassandra.yaml to 1.
> {code:java}
> // cassandra.yaml
> column_index_size_in_kb: 1 {code}
> 2. Execute the following commands (I masked out all data in the INSERT
> command using a long string of 'A' to make it simpler)
> {code:java}
> CREATE KEYSPACE ks WITH REPLICATION = { 'class' : 'SimpleStrategy',
> 'replication_factor' : 1 };
> CREATE TABLE ks.tb (c3 INT, c7 INT, c0 TEXT,c6 set<TEXT>, PRIMARY KEY (c3,
> c7, c0));
> DELETE FROM ks.tb WHERE c3 = 1 AND c7 = 1;
> DELETE FROM ks.tb WHERE c3 = 1 AND c7 = 1 AND c0 = 'e';
> INSERT INTO ks.tb (c7, c3, c6, c0) VALUES
> (1,1,{'AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA','AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA','AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA'},'AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA');
> {code}
> Perform a read in the old version, it works normally
> {code:java}
> cqlsh> SELECT c7, c0, c3 FROM ks.tb WHERE c3 = 1 AND c7 = 1 ORDER BY c7 DESC;
> c7 | c0 | c3
> ----+--------------------------------------------------+----
> 1 | AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA | 1
> (1 rows) {code}
> 3. Drain and stop 2.2.19.
> {code:java}
> bin/nodetool -h ::FFFF:127.0.0.1 drain
> bin/nodetool -h ::FFFF:127.0.0.1 stopdaemon; {code}
> 4. Copy data into 3.0.30 and start up (using default config is okay), read
> the legacy data with the following read command
> {code:java}
> SELECT c7, c0, c3 FROM ks.tb WHERE c3 = 1 AND c7 = 1 ORDER BY c7 DESC; {code}
> It would fail and throw an error in system.log.
> {code:java}
> ➜ apache-cassandra-3.0.30 bin/cqlsh
> Connected to Test Cluster at 127.0.0.1:9042.
> [cqlsh 5.0.1 | Cassandra 3.0.30-SNAPSHOT | CQL spec 3.4.0 | Native protocol
> v4]
> Use HELP for help.
> cqlsh> SELECT c7, c0, c3 FROM ks.tb WHERE c3 = 1 AND c7 = 1 ORDER BY c7 DESC;
> ReadFailure: Error from server: code=1300 [Replica(s) failed to execute read]
> message="Operation failed - received 0 responses and 1 failures"
> info={'failures': 1, 'received_responses': 0, 'required_responses': 1,
> 'consistency': 'ONE'} {code}
> system.log
> {code:java}
> INFO [main] 2024-05-08 00:52:11,649 CassandraDaemon.java:653 - Startup
> complete
> ERROR [SharedPool-Worker-2] 2024-05-08 00:52:13,695
> AbstractLocalAwareExecutorService.java:166 - Uncaught exception on thread
> Thread[SharedPool-Worker-2,10,main]
> java.lang.RuntimeException: java.lang.IllegalStateException: SSTABLE
> UnfilteredRowIterator for ks.tb has an illegal RT bounds sequence: unexpected
> end bound or boundary Marker INCL_START_BOUND(1)@1715129489466439/1715129489
> at
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2656)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:162)
> at
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:134)
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:109)
> at java.lang.Thread.run(Thread.java:750)
> Caused by: java.lang.IllegalStateException: SSTABLE UnfilteredRowIterator for
> ks.tb has an illegal RT bounds sequence: unexpected end bound or boundary
> Marker INCL_START_BOUND(1)@1715129489466439/1715129489
> at
> org.apache.cassandra.db.transform.RTBoundValidator$RowsTransformation.ise(RTBoundValidator.java:120)
> at
> org.apache.cassandra.db.transform.RTBoundValidator$RowsTransformation.applyToMarker(RTBoundValidator.java:87)
> at
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:144)
> at
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:129)
> at
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:131)
> at
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:87)
> at
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:77)
> at
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:297)
> at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.build(ReadResponse.java:187)
> at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.<init>(ReadResponse.java:180)
> at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.<init>(ReadResponse.java:176)
> at
> org.apache.cassandra.db.ReadResponse.createDataResponse(ReadResponse.java:76)
> at
> org.apache.cassandra.db.ReadCommand.createResponse(ReadCommand.java:347)
> at
> org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:1914)
> at org.apache.cassandra.service {code}
> I attached the data.tar.gz file generated from the 2.2.19, using 3.0.30 to
> start with it and execute the above read command can directly reproduce it.
> I also attached the cassandra.yaml file used in 2.2.19.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
