schema IAE and read path NPE after cluster re-deploy ----------------------------------------------------
Key: CASSANDRA-3932 URL: https://issues.apache.org/jira/browse/CASSANDRA-3932 Project: Cassandra Issue Type: Bug Components: Core Reporter: Peter Schuller Assignee: Peter Schuller Fix For: 1.1.0 On the same cluster (but later) as the one where we observed CASSANDRA-3931 we were running some performance/latency testing. ycsb reads, plus a separate little python client. All was fine. I then did a fast re-deploy for changed GC settings, which would have let to a complete cluster restart almost simultaneously (triggering races?). When I re-ran my Python client, I suddenly got an error saying Keyspace1 did not exist. On re-run I started getting timeouts. Looking at the endpoints of the key that I was getting a timeout for, the first error ever seen is: {code} java.lang.IllegalArgumentException: Unknown ColumnFamily Standard1 in keyspace Keyspace1 at org.apache.cassandra.config.Schema.getComparator(Schema.java:234) at org.apache.cassandra.db.ColumnFamily.getComparatorFor(ColumnFamily.java:312) at org.apache.cassandra.db.ReadCommand.getComparator(ReadCommand.java:94) at org.apache.cassandra.db.SliceByNamesReadCommand.<init>(SliceByNamesReadCommand.java:44) at org.apache.cassandra.db.SliceByNamesReadCommandSerializer.deserialize(SliceByNamesReadCommand.java:113) at org.apache.cassandra.db.SliceByNamesReadCommandSerializer.deserialize(SliceByNamesReadCommand.java:81) at org.apache.cassandra.db.ReadCommandSerializer.deserialize(ReadCommand.java:134) at org.apache.cassandra.db.ReadVerbHandler.doVerb(ReadVerbHandler.java:53) at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:59) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) {code} And later in the read path NPE:s like these: {code} java.lang.NullPointerException at org.apache.cassandra.db.Table.createReplicationStrategy(Table.java:321) at org.apache.cassandra.db.Table.<init>(Table.java:277) at org.apache.cassandra.db.Table.open(Table.java:120) at org.apache.cassandra.db.Table.open(Table.java:103) at org.apache.cassandra.db.ReadVerbHandler.doVerb(ReadVerbHandler.java:54) at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:59) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) {code} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira