A static collection is probably atypical, and again, would encourage you to open a JIRA.
This seems like a case we should be able to find in a simulator. On Tue, Dec 12, 2023 at 2:05 PM Sebastian Marsching <sebast...@marsching.com> wrote: > I assume these are column names of a non-system table. > > This is correct. It is one of our application tables. The table has the > following schema: > > CREATE TABLE pv_archive.channels ( > channel_name text, > decimation_level int, > bucket_start_time bigint, > channel_data_id uuid static, > control_system_type text static, > server_id uuid static, > decimation_levels set<int> static, > bucket_end_time bigint, > PRIMARY KEY (channel_name, decimation_level, bucket_start_time) > ) WITH CLUSTERING ORDER BY (decimation_level ASC, bucket_start_time ASC) > AND additional_write_policy = '99p' > AND bloom_filter_fp_chance = 0.01 > AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'} > AND cdc = false > AND comment = '' > AND compaction = {'class': > 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', > 'max_threshold': '32', 'min_threshold': '4'} > AND compression = {'chunk_length_in_kb': '64', 'class': > 'org.apache.cassandra.io.compress.LZ4Compressor'} > AND memtable = 'default' > AND crc_check_chance = 1.0 > AND default_time_to_live = 0 > AND extensions = {} > AND gc_grace_seconds = 2592000 > AND max_index_interval = 2048 > AND memtable_flush_period_in_ms = 0 > AND min_index_interval = 128 > AND read_repair = 'BLOCKING' > AND speculative_retry = '99p'; > > From the stack trace, this looks like an error from a node which was > running 4.1.3, and this node was not the coordinator for this query. > > I did some research and found these bug reports which may be related: > > - CASSANDRA-15899 > <https://issues.apache.org/jira/browse/CASSANDRA-15899> Dropping a > column can break queries until the schema is fully propagated > - CASSANDRA-16735 > <https://issues.apache.org/jira/browse/CASSANDRA-16735> Adding columns > via ALTER TABLE can generate corrupt sstables > > The solution for CASSANDRA-16735 was to revert CASSANDRA-15899, according > to the comments in the ticket. > > This does look like CASSANDRA-15899 is back, but I can't see why it was > only happening when the nodes were running mixed versions, and then stopped > after all nodes were upgraded. > > I do not think that it is either of these bugs. These bugs occurred after > altering a table, but I can say with certainty that this table has never > been altered after it was created years ago. > > It must be a very strange bug where C* somehow gets confused about the > schema for a table during an upgrade, even when the schema for this table > did not change. I wonder whether it might have anything to do with the use > of static columns… > > We have a second cluster that is using a setup that is pretty much > identical and that we have not upgraded yet. We are now scheduling a bit of > downtime for the upgrade there. As that cluster is rather small (only six > nodes), upgrading the whole cluster should not take very long. > > It will be interesting to see, whether the problem will appear there to. > If it doesn’t, this might have been some kind of freak accident that might > not warrant further investigation. If it happens again, I might be able to > collect more information. > >