[jira] [Created] (CASSANDRA-16257) When upgrading from 2.1.0 to 2.2.0, UnknownColumnFamilyException encountered on upgraded nodes

2020-11-09 Thread Zhuqi Jin (Jira)
Zhuqi Jin created CASSANDRA-16257:
-

 Summary: When upgrading from 2.1.0 to 2.2.0, 
UnknownColumnFamilyException encountered on upgraded nodes
 Key: CASSANDRA-16257
 URL: https://issues.apache.org/jira/browse/CASSANDRA-16257
 Project: Cassandra
  Issue Type: Bug
Reporter: Zhuqi Jin


Both using 2.1.0 node as seed and using 2.2.0 node as seed will fail with this 
similar error:
{code:java}
org.apache.cassandra.db.UnknownColumnFamilyException: Couldn't find 
cfId=49a44701-d6bc-11ea-848f-4766f428c026 (when using 2.1.0 node as seed)
org.apache.cassandra.db.UnknownColumnFamilyException: Couldn't find 
cfId=727d9450-d6c7-11ea-9370-4766f428c026 (when using 2.2.0 node as seed)at 
org.apache.cassandra.db.ColumnFamilySerializer.deserializeCfId(ColumnFamilySerializer.java:164)
 ~[apache-cassandra-2.1.0.jar:2.1.0]at 
org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:97)
 ~[apache-cassandra-2.1.0.jar:2.1.0]at 
org.apache.cassandra.db.Mutation$MutationSerializer.deserializeOneCf(Mutation.java:322)
 ~[apache-cassandra-2.1.0.jar:2.1.0]at 
org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:302)
 ~[apache-cassandra-2.1.0.jar:2.1.0]at 
org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:330)
 ~[apache-cassandra-2.1.0.jar:2.1.0]at 
org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:272)
 ~[apache-cassandra-2.1.0.jar:2.1.0]at 
org.apache.cassandra.net.MessageIn.read(MessageIn.java:99) 
~[apache-cassandra-2.1.0.jar:2.1.0]at 
org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:165)
 ~[apache-cassandra-2.1.0.jar:2.1.0]at 
org.apache.cassandra.net.IncomingTcpConnection.handleModernVersion(IncomingTcpConnection.java:147)
 ~[apache-cassandra-2.1.0.jar:2.1.0]at 
org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:82)
 ~[apache-cassandra-2.1.0.jar:2.1.0]{code}
This seems to originate from the difference in the settings in the stress 
tests, and is not a data format bug. The field `cfId` is a UUID string 
generated from another two fields: `ksName` and `cfName`. These two names are 
not the interior settings in the cluster, instead they come from user 
specification.

In 2.1.0, the stress test configures these two names to be “Keyspace1” and 
“Standard1”. See 
[https://github.com/apache/cassandra/blob/cassandra-2.1.0/tools/stress/src/org/apache/cassandra/stress/settings/SettingsSchema.java#L84]
 . Here is my log:
{code:java}
WARN  [MigrationStage:1] 2020-08-05 03:17:20,514 Schema.java:330 - Schema.load: 
Adding org.apache.cassandra.config.CFMetaData@6299796f…..., key = 
(Keyspace1,Standard1), val = 49a44701-d6bc-11ea-848f-4766f428c026
{code}
In 2.2.0, the stress test configures these two names to be “keyspace1” and 
“standard1” (note that the names are case-sensitive). See 
https://github.com/apache/cassandra/blob/cassandra-2.2.0/tools/stress/src/org/apache/cassandra/stress/settings/SettingsSchema.java#L225
  . Here is my log:
{code:java}
WARN  [MigrationStage:1] 2020-08-05 02:57:49,680 Schema.java:350 - Schema.load: 
Adding org.apache.cassandra.config.CFMetaData@719822c2……..., key = 
(keyspace1,standard1), val = 727d9450-d6c7-11ea-9370-4766f428c026
{code}
When a 2.2.0 message reaches a 2.1.0 node, or vice versa, the receiver will 
look up the `cfId` field in a map. Since the key is not present in its own map, 
the look up will fail with the above message. Unluckily this problem is quite 
hard to fix, since many other places also depend on the name. So I can not 
overcome this problem and find actual data format bugs between these two 
versions currently.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Comment Edited] (CASSANDRA-15794) Upgraded C* (4.x) fail to start because of Compact Tables & dropping compact tables in downgraded C* (3.11.4) introduces non-existent columns

2020-06-25 Thread Zhuqi Jin (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17134575#comment-17134575
 ] 

Zhuqi Jin edited comment on CASSANDRA-15794 at 6/25/20, 2:53 PM:
-

Hi, [~ifesdjeen].

I created patches for 3.0 and 3.11, according to the third method we discussed 
earlier.

[^CASSANDRA-15794-branch-3.0.patch]

 

I'd like to move on to the second point and make sure we don't write any commit 
log messages in such case. Could you help give me some pointers of where commit 
logs are generated in this scenario?


was (Author: zhuqi1108):
Hi, [~ifesdjeen].

I created patches for 3.0 and 3.11, according to the third method we discussed 
earlier.

[^CASSANDRA-15794-branch-3.0.patch]

I've attached the patches. Would you mind reviewing them? 

And I'd like to move on to the second method. Could you please do me a favor? 
We don‘t want to generate new commit logs before we hit the error in 4.x, so I 
need to know when and where the commit logs were written.

> Upgraded C* (4.x) fail to start because of Compact Tables & dropping compact 
> tables in downgraded C* (3.11.4) introduces non-existent columns
> -
>
> Key: CASSANDRA-15794
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15794
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Zhuqi Jin
>Priority: Normal
> Attachments: CASSANDRA-15794-branch-3.0.patch, 
> CASSANDRA-15794-branch-3.11.patch
>
>
> We tried to test upgrading a 3.11.4 C* cluster to 4.x and run into the 
> following problems. 
>  * We started a single 3.11.4 C* node. 
>  * We ran cassandra-stress like this
> {code:java}
> ./cassandra-stress write n = 30 -rate threads = 10 -node  172.17.0.2 {code}
>  * We stopped this node, and started a C* node running C* compiled from trunk 
> (git commit: e394dc0bb32f612a476269010930c617dd1ed3cb)
>  * New C* failed to start with the following error message
> {code:java}
> ERROR [main] 2020-05-07 00:58:18,503 CassandraDaemon.java:245 - Error while 
> loading schema: ERROR [main] 2020-05-07 00:58:18,503 CassandraDaemon.java:245 
> - Error while loading schema: java.lang.IllegalArgumentException: Compact 
> Tables are not allowed in Cassandra starting with 4.0 version. Use `ALTER ... 
> DROP COMPACT STORAGE` command supplied in 3.x/3.11 Cassandra in order to 
> migrate off Compact Storage. at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchTable(SchemaKeyspace.java:965)
>  at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchTables(SchemaKeyspace.java:924)
>  at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:883)
>  at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:874)
>  at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:862)
>  at org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:102) at 
> org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:91) at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:241) 
> at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:653)
>  at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:770)Exception
>  (java.lang.IllegalArgumentException) encountered during startup: Compact 
> Tables are not allowed in Cassandra starting with 4.0 version. Use `ALTER ... 
> DROP COMPACT STORAGE` command supplied in 3.x/3.11 Cassandra in order to 
> migrate off Compact Storage.ERROR [main] 2020-05-07 00:58:18,520 
> CassandraDaemon.java:792 - Exception encountered during 
> startupjava.lang.IllegalArgumentException: Compact Tables are not allowed in 
> Cassandra starting with 4.0 version. Use `ALTER ... DROP COMPACT STORAGE` 
> command supplied in 3.x/3.11 Cassandra in order to migrate off Compact 
> Storage. at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchTable(SchemaKeyspace.java:965)
>  at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchTables(SchemaKeyspace.java:924)
>  at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:883)
>  at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:874)
>  at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:862)
>  at org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:102) at 
> org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:91) at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:241) 
> at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:653)
>  at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:770){code}
>  * We stopped the 

[jira] [Comment Edited] (CASSANDRA-15794) Upgraded C* (4.x) fail to start because of Compact Tables & dropping compact tables in downgraded C* (3.11.4) introduces non-existent columns

2020-06-16 Thread Zhuqi Jin (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17134575#comment-17134575
 ] 

Zhuqi Jin edited comment on CASSANDRA-15794 at 6/16/20, 7:57 PM:
-

Hi, [~ifesdjeen].

I created patches for 3.0 and 3.11, according to the third method we discussed 
earlier.

[^CASSANDRA-15794-branch-3.0.patch]

I've attached the patches. Would you mind reviewing them? 

And I'd like to move on to the second method. Could you please do me a favor? 
We don‘t want to generate new commit logs before we hit the error in 4.x, so I 
need to know when and where the commit logs were written.


was (Author: zhuqi1108):
Hi, [~ifesdjeen].

I created patches for 3.0 and 3.11, according to the third method we discussed 
earlier.

[^CASSANDRA-15794-branch-3.0.patch]

I've attached the patches. Would you mind reviewing them? 

> Upgraded C* (4.x) fail to start because of Compact Tables & dropping compact 
> tables in downgraded C* (3.11.4) introduces non-existent columns
> -
>
> Key: CASSANDRA-15794
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15794
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Zhuqi Jin
>Priority: Normal
> Attachments: CASSANDRA-15794-branch-3.0.patch, 
> CASSANDRA-15794-branch-3.11.patch
>
>
> We tried to test upgrading a 3.11.4 C* cluster to 4.x and run into the 
> following problems. 
>  * We started a single 3.11.4 C* node. 
>  * We ran cassandra-stress like this
> {code:java}
> ./cassandra-stress write n = 30 -rate threads = 10 -node  172.17.0.2 {code}
>  * We stopped this node, and started a C* node running C* compiled from trunk 
> (git commit: e394dc0bb32f612a476269010930c617dd1ed3cb)
>  * New C* failed to start with the following error message
> {code:java}
> ERROR [main] 2020-05-07 00:58:18,503 CassandraDaemon.java:245 - Error while 
> loading schema: ERROR [main] 2020-05-07 00:58:18,503 CassandraDaemon.java:245 
> - Error while loading schema: java.lang.IllegalArgumentException: Compact 
> Tables are not allowed in Cassandra starting with 4.0 version. Use `ALTER ... 
> DROP COMPACT STORAGE` command supplied in 3.x/3.11 Cassandra in order to 
> migrate off Compact Storage. at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchTable(SchemaKeyspace.java:965)
>  at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchTables(SchemaKeyspace.java:924)
>  at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:883)
>  at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:874)
>  at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:862)
>  at org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:102) at 
> org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:91) at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:241) 
> at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:653)
>  at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:770)Exception
>  (java.lang.IllegalArgumentException) encountered during startup: Compact 
> Tables are not allowed in Cassandra starting with 4.0 version. Use `ALTER ... 
> DROP COMPACT STORAGE` command supplied in 3.x/3.11 Cassandra in order to 
> migrate off Compact Storage.ERROR [main] 2020-05-07 00:58:18,520 
> CassandraDaemon.java:792 - Exception encountered during 
> startupjava.lang.IllegalArgumentException: Compact Tables are not allowed in 
> Cassandra starting with 4.0 version. Use `ALTER ... DROP COMPACT STORAGE` 
> command supplied in 3.x/3.11 Cassandra in order to migrate off Compact 
> Storage. at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchTable(SchemaKeyspace.java:965)
>  at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchTables(SchemaKeyspace.java:924)
>  at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:883)
>  at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:874)
>  at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:862)
>  at org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:102) at 
> org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:91) at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:241) 
> at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:653)
>  at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:770){code}
>  * We stopped the trunk version C* and started the 3.11.4 version C*. 
>  * 3.11.4 C* failed to start with the following error messages: 
> {code:java}
> 

[jira] [Commented] (CASSANDRA-15794) Upgraded C* (4.x) fail to start because of Compact Tables & dropping compact tables in downgraded C* (3.11.4) introduces non-existent columns

2020-06-12 Thread Zhuqi Jin (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17134575#comment-17134575
 ] 

Zhuqi Jin commented on CASSANDRA-15794:
---

Hi, [~ifesdjeen].

I created patches for 3.0 and 3.11, according to the third method we discussed 
earlier.

[^CASSANDRA-15794-branch-3.0.patch]

I've attached the patches. Would you mind reviewing them? 

> Upgraded C* (4.x) fail to start because of Compact Tables & dropping compact 
> tables in downgraded C* (3.11.4) introduces non-existent columns
> -
>
> Key: CASSANDRA-15794
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15794
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Zhuqi Jin
>Priority: Normal
> Attachments: CASSANDRA-15794-branch-3.0.patch, 
> CASSANDRA-15794-branch-3.11.patch
>
>
> We tried to test upgrading a 3.11.4 C* cluster to 4.x and run into the 
> following problems. 
>  * We started a single 3.11.4 C* node. 
>  * We ran cassandra-stress like this
> {code:java}
> ./cassandra-stress write n = 30 -rate threads = 10 -node  172.17.0.2 {code}
>  * We stopped this node, and started a C* node running C* compiled from trunk 
> (git commit: e394dc0bb32f612a476269010930c617dd1ed3cb)
>  * New C* failed to start with the following error message
> {code:java}
> ERROR [main] 2020-05-07 00:58:18,503 CassandraDaemon.java:245 - Error while 
> loading schema: ERROR [main] 2020-05-07 00:58:18,503 CassandraDaemon.java:245 
> - Error while loading schema: java.lang.IllegalArgumentException: Compact 
> Tables are not allowed in Cassandra starting with 4.0 version. Use `ALTER ... 
> DROP COMPACT STORAGE` command supplied in 3.x/3.11 Cassandra in order to 
> migrate off Compact Storage. at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchTable(SchemaKeyspace.java:965)
>  at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchTables(SchemaKeyspace.java:924)
>  at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:883)
>  at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:874)
>  at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:862)
>  at org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:102) at 
> org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:91) at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:241) 
> at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:653)
>  at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:770)Exception
>  (java.lang.IllegalArgumentException) encountered during startup: Compact 
> Tables are not allowed in Cassandra starting with 4.0 version. Use `ALTER ... 
> DROP COMPACT STORAGE` command supplied in 3.x/3.11 Cassandra in order to 
> migrate off Compact Storage.ERROR [main] 2020-05-07 00:58:18,520 
> CassandraDaemon.java:792 - Exception encountered during 
> startupjava.lang.IllegalArgumentException: Compact Tables are not allowed in 
> Cassandra starting with 4.0 version. Use `ALTER ... DROP COMPACT STORAGE` 
> command supplied in 3.x/3.11 Cassandra in order to migrate off Compact 
> Storage. at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchTable(SchemaKeyspace.java:965)
>  at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchTables(SchemaKeyspace.java:924)
>  at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:883)
>  at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:874)
>  at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:862)
>  at org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:102) at 
> org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:91) at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:241) 
> at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:653)
>  at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:770){code}
>  * We stopped the trunk version C* and started the 3.11.4 version C*. 
>  * 3.11.4 C* failed to start with the following error messages: 
> {code:java}
> Exception (java.lang.IllegalStateException) encountered during startup: 
> Unknown commitlog version 7Exception (java.lang.IllegalStateException) 
> encountered during startup: Unknown commitlog version 7ERROR [main] 
> 2020-05-07 01:13:26,798 CassandraDaemon.java:749 - Exception encountered 
> during startupjava.lang.IllegalStateException: Unknown commitlog version 7 at 
> org.apache.cassandra.db.commitlog.CommitLogDescriptor.getMessagingVersion(CommitLogDescriptor.java:227)
>  ~[main/:na] at 
> 

[jira] [Updated] (CASSANDRA-15794) Upgraded C* (4.x) fail to start because of Compact Tables & dropping compact tables in downgraded C* (3.11.4) introduces non-existent columns

2020-06-12 Thread Zhuqi Jin (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhuqi Jin updated CASSANDRA-15794:
--
Attachment: CASSANDRA-15794-branch-3.11.patch
CASSANDRA-15794-branch-3.0.patch

> Upgraded C* (4.x) fail to start because of Compact Tables & dropping compact 
> tables in downgraded C* (3.11.4) introduces non-existent columns
> -
>
> Key: CASSANDRA-15794
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15794
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Zhuqi Jin
>Priority: Normal
> Attachments: CASSANDRA-15794-branch-3.0.patch, 
> CASSANDRA-15794-branch-3.11.patch
>
>
> We tried to test upgrading a 3.11.4 C* cluster to 4.x and run into the 
> following problems. 
>  * We started a single 3.11.4 C* node. 
>  * We ran cassandra-stress like this
> {code:java}
> ./cassandra-stress write n = 30 -rate threads = 10 -node  172.17.0.2 {code}
>  * We stopped this node, and started a C* node running C* compiled from trunk 
> (git commit: e394dc0bb32f612a476269010930c617dd1ed3cb)
>  * New C* failed to start with the following error message
> {code:java}
> ERROR [main] 2020-05-07 00:58:18,503 CassandraDaemon.java:245 - Error while 
> loading schema: ERROR [main] 2020-05-07 00:58:18,503 CassandraDaemon.java:245 
> - Error while loading schema: java.lang.IllegalArgumentException: Compact 
> Tables are not allowed in Cassandra starting with 4.0 version. Use `ALTER ... 
> DROP COMPACT STORAGE` command supplied in 3.x/3.11 Cassandra in order to 
> migrate off Compact Storage. at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchTable(SchemaKeyspace.java:965)
>  at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchTables(SchemaKeyspace.java:924)
>  at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:883)
>  at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:874)
>  at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:862)
>  at org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:102) at 
> org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:91) at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:241) 
> at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:653)
>  at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:770)Exception
>  (java.lang.IllegalArgumentException) encountered during startup: Compact 
> Tables are not allowed in Cassandra starting with 4.0 version. Use `ALTER ... 
> DROP COMPACT STORAGE` command supplied in 3.x/3.11 Cassandra in order to 
> migrate off Compact Storage.ERROR [main] 2020-05-07 00:58:18,520 
> CassandraDaemon.java:792 - Exception encountered during 
> startupjava.lang.IllegalArgumentException: Compact Tables are not allowed in 
> Cassandra starting with 4.0 version. Use `ALTER ... DROP COMPACT STORAGE` 
> command supplied in 3.x/3.11 Cassandra in order to migrate off Compact 
> Storage. at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchTable(SchemaKeyspace.java:965)
>  at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchTables(SchemaKeyspace.java:924)
>  at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:883)
>  at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:874)
>  at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:862)
>  at org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:102) at 
> org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:91) at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:241) 
> at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:653)
>  at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:770){code}
>  * We stopped the trunk version C* and started the 3.11.4 version C*. 
>  * 3.11.4 C* failed to start with the following error messages: 
> {code:java}
> Exception (java.lang.IllegalStateException) encountered during startup: 
> Unknown commitlog version 7Exception (java.lang.IllegalStateException) 
> encountered during startup: Unknown commitlog version 7ERROR [main] 
> 2020-05-07 01:13:26,798 CassandraDaemon.java:749 - Exception encountered 
> during startupjava.lang.IllegalStateException: Unknown commitlog version 7 at 
> org.apache.cassandra.db.commitlog.CommitLogDescriptor.getMessagingVersion(CommitLogDescriptor.java:227)
>  ~[main/:na] at 
> org.apache.cassandra.db.commitlog.CommitLogReader.shouldSkipSegmentId(CommitLogReader.java:276)
>  ~[main/:na] at 
> 

[jira] [Commented] (CASSANDRA-15794) Upgraded C* (4.x) fail to start because of Compact Tables & dropping compact tables in downgraded C* (3.11.4) introduces non-existent columns

2020-05-18 Thread Zhuqi Jin (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17110435#comment-17110435
 ] 

Zhuqi Jin commented on CASSANDRA-15794:
---

Thanks for you prompt reply.

I still think that users who have already used compact storage should be 
provided with a solution (even assuming CASSANDRA-15811 is fixed). Otherwise, 
users may be confused, especially when they roll it back but cannot start 3.11.

For example,

1) Instead of directly block cassandra startup, is it possible to correctly 
drop existing compact storage in 4.x (with the fix from CASSANDRA-15811)?

2) Or, if we stick to ask users to downgrade to 3.x and drop compact storage, 
can we not generate new commit log before we hit the error in 4.x, so that we 
avoid blocking 3.x from starting?

3) Or, block or warn users when they try to introduce compact storage in 3.x? 
Throw a warning telling them that compact storage is deprecated in 3.x and will 
cause problem upgrading to 4.x, or simply block them from doing that, or at 
least modify the manual of 3.X, indicating that you should not use compact 
storage.

 

If you think any of these is reasonable, I can provide help.

 

> Upgraded C* (4.x) fail to start because of Compact Tables & dropping compact 
> tables in downgraded C* (3.11.4) introduces non-existent columns
> -
>
> Key: CASSANDRA-15794
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15794
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Zhuqi Jin
>Priority: Normal
>
> We tried to test upgrading a 3.11.4 C* cluster to 4.x and run into the 
> following problems. 
>  * We started a single 3.11.4 C* node. 
>  * We ran cassandra-stress like this
> {code:java}
> ./cassandra-stress write n = 30 -rate threads = 10 -node  172.17.0.2 {code}
>  * We stopped this node, and started a C* node running C* compiled from trunk 
> (git commit: e394dc0bb32f612a476269010930c617dd1ed3cb)
>  * New C* failed to start with the following error message
> {code:java}
> ERROR [main] 2020-05-07 00:58:18,503 CassandraDaemon.java:245 - Error while 
> loading schema: ERROR [main] 2020-05-07 00:58:18,503 CassandraDaemon.java:245 
> - Error while loading schema: java.lang.IllegalArgumentException: Compact 
> Tables are not allowed in Cassandra starting with 4.0 version. Use `ALTER ... 
> DROP COMPACT STORAGE` command supplied in 3.x/3.11 Cassandra in order to 
> migrate off Compact Storage. at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchTable(SchemaKeyspace.java:965)
>  at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchTables(SchemaKeyspace.java:924)
>  at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:883)
>  at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:874)
>  at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:862)
>  at org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:102) at 
> org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:91) at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:241) 
> at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:653)
>  at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:770)Exception
>  (java.lang.IllegalArgumentException) encountered during startup: Compact 
> Tables are not allowed in Cassandra starting with 4.0 version. Use `ALTER ... 
> DROP COMPACT STORAGE` command supplied in 3.x/3.11 Cassandra in order to 
> migrate off Compact Storage.ERROR [main] 2020-05-07 00:58:18,520 
> CassandraDaemon.java:792 - Exception encountered during 
> startupjava.lang.IllegalArgumentException: Compact Tables are not allowed in 
> Cassandra starting with 4.0 version. Use `ALTER ... DROP COMPACT STORAGE` 
> command supplied in 3.x/3.11 Cassandra in order to migrate off Compact 
> Storage. at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchTable(SchemaKeyspace.java:965)
>  at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchTables(SchemaKeyspace.java:924)
>  at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:883)
>  at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:874)
>  at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:862)
>  at org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:102) at 
> org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:91) at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:241) 
> at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:653)
>  at 
> 

[jira] [Updated] (CASSANDRA-15794) Upgraded C* (4.x) fail to start because of Compact Tables & dropping compact tables in downgraded C* (3.11.4) introduces non-existent columns

2020-05-16 Thread Zhuqi Jin (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhuqi Jin updated CASSANDRA-15794:
--
Description: 
We tried to test upgrading a 3.11.4 C* cluster to 4.x and run into the 
following problems. 
 * We started a single 3.11.4 C* node. 
 * We ran cassandra-stress like this

{code:java}
./cassandra-stress write n = 30 -rate threads = 10 -node  172.17.0.2 {code}
 * We stopped this node, and started a C* node running C* compiled from trunk 
(git commit: e394dc0bb32f612a476269010930c617dd1ed3cb)
 * New C* failed to start with the following error message

{code:java}
ERROR [main] 2020-05-07 00:58:18,503 CassandraDaemon.java:245 - Error while 
loading schema: ERROR [main] 2020-05-07 00:58:18,503 CassandraDaemon.java:245 - 
Error while loading schema: java.lang.IllegalArgumentException: Compact Tables 
are not allowed in Cassandra starting with 4.0 version. Use `ALTER ... DROP 
COMPACT STORAGE` command supplied in 3.x/3.11 Cassandra in order to migrate off 
Compact Storage. at 
org.apache.cassandra.schema.SchemaKeyspace.fetchTable(SchemaKeyspace.java:965) 
at 
org.apache.cassandra.schema.SchemaKeyspace.fetchTables(SchemaKeyspace.java:924) 
at 
org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:883)
 at 
org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:874)
 at 
org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:862)
 at org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:102) at 
org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:91) at 
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:241) at 
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:653) 
at 
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:770)Exception
 (java.lang.IllegalArgumentException) encountered during startup: Compact 
Tables are not allowed in Cassandra starting with 4.0 version. Use `ALTER ... 
DROP COMPACT STORAGE` command supplied in 3.x/3.11 Cassandra in order to 
migrate off Compact Storage.ERROR [main] 2020-05-07 00:58:18,520 
CassandraDaemon.java:792 - Exception encountered during 
startupjava.lang.IllegalArgumentException: Compact Tables are not allowed in 
Cassandra starting with 4.0 version. Use `ALTER ... DROP COMPACT STORAGE` 
command supplied in 3.x/3.11 Cassandra in order to migrate off Compact Storage. 
at 
org.apache.cassandra.schema.SchemaKeyspace.fetchTable(SchemaKeyspace.java:965) 
at 
org.apache.cassandra.schema.SchemaKeyspace.fetchTables(SchemaKeyspace.java:924) 
at 
org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:883)
 at 
org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:874)
 at 
org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:862)
 at org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:102) at 
org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:91) at 
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:241) at 
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:653) 
at 
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:770){code}
 * We stopped the trunk version C* and started the 3.11.4 version C*. 
 * 3.11.4 C* failed to start with the following error messages: 

{code:java}
Exception (java.lang.IllegalStateException) encountered during startup: Unknown 
commitlog version 7Exception (java.lang.IllegalStateException) encountered 
during startup: Unknown commitlog version 7ERROR [main] 2020-05-07 01:13:26,798 
CassandraDaemon.java:749 - Exception encountered during 
startupjava.lang.IllegalStateException: Unknown commitlog version 7 at 
org.apache.cassandra.db.commitlog.CommitLogDescriptor.getMessagingVersion(CommitLogDescriptor.java:227)
 ~[main/:na] at 
org.apache.cassandra.db.commitlog.CommitLogReader.shouldSkipSegmentId(CommitLogReader.java:276)
 ~[main/:na] at 
org.apache.cassandra.db.commitlog.CommitLogReader.readCommitLogSegment(CommitLogReader.java:220)
 ~[main/:na] at 
org.apache.cassandra.db.commitlog.CommitLogReader.readAllFiles(CommitLogReader.java:132)
 ~[main/:na] at 
org.apache.cassandra.db.commitlog.CommitLogReplayer.replayFiles(CommitLogReplayer.java:137)
 ~[main/:na] at 
org.apache.cassandra.db.commitlog.CommitLog.recoverFiles(CommitLog.java:167) 
~[main/:na] at 
org.apache.cassandra.db.commitlog.CommitLog.recoverSegmentsOnDisk(CommitLog.java:148)
 ~[main/:na] at 
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:324) 
[main/:na] at 
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:620) 
[main/:na] at 
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:732) 
[main/:na]
{code}
We manually deleted all new version commit logs and it worked. However, it's 

[jira] [Updated] (CASSANDRA-15794) Upgraded C* (4.x) fail to start because of Compact Tables & dropping compact tables in downgraded C* (3.11.4) introduces non-existent columns

2020-05-16 Thread Zhuqi Jin (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhuqi Jin updated CASSANDRA-15794:
--
Discovered By: User Report
 Severity: Normal
Since Version: 3.11.4
  Description: 
We tried to test upgrading a 3.11.4 C* cluster to 4.x and run into the 
following problems. 
 * We started a single 3.11.4 C* node. 
 * We ran cassandra-stress like this

{code:java}
./cassandra-stress write n = 30 -rate threads = 10 -node  172.17.0.2 {code}
 * We stopped this node, and started a C* node running C* compiled from trunk 
(git commit: e394dc0bb32f612a476269010930c617dd1ed3cb)
 * New C* failed to start with the following error message

{code:java}
ERROR [main] 2020-05-07 00:58:18,503 CassandraDaemon.java:245 - Error while 
loading schema: ERROR [main] 2020-05-07 00:58:18,503 CassandraDaemon.java:245 - 
Error while loading schema: java.lang.IllegalArgumentException: Compact Tables 
are not allowed in Cassandra starting with 4.0 version. Use `ALTER ... DROP 
COMPACT STORAGE` command supplied in 3.x/3.11 Cassandra in order to migrate off 
Compact Storage. at 
org.apache.cassandra.schema.SchemaKeyspace.fetchTable(SchemaKeyspace.java:965) 
at 
org.apache.cassandra.schema.SchemaKeyspace.fetchTables(SchemaKeyspace.java:924) 
at 
org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:883)
 at 
org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:874)
 at 
org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:862)
 at org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:102) at 
org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:91) at 
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:241) at 
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:653) 
at 
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:770)Exception
 (java.lang.IllegalArgumentException) encountered during startup: Compact 
Tables are not allowed in Cassandra starting with 4.0 version. Use `ALTER ... 
DROP COMPACT STORAGE` command supplied in 3.x/3.11 Cassandra in order to 
migrate off Compact Storage.ERROR [main] 2020-05-07 00:58:18,520 
CassandraDaemon.java:792 - Exception encountered during 
startupjava.lang.IllegalArgumentException: Compact Tables are not allowed in 
Cassandra starting with 4.0 version. Use `ALTER ... DROP COMPACT STORAGE` 
command supplied in 3.x/3.11 Cassandra in order to migrate off Compact Storage. 
at 
org.apache.cassandra.schema.SchemaKeyspace.fetchTable(SchemaKeyspace.java:965) 
at 
org.apache.cassandra.schema.SchemaKeyspace.fetchTables(SchemaKeyspace.java:924) 
at 
org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:883)
 at 
org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:874)
 at 
org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:862)
 at org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:102) at 
org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:91) at 
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:241) at 
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:653) 
at 
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:770){code}
 * We stopped the trunk version C* and started the 3.11.4 version C*. 
 * 3.11.4 C* failed to start with the following error messages: 

{code:java}
Exception (java.lang.IllegalStateException) encountered during startup: Unknown 
commitlog version 7Exception (java.lang.IllegalStateException) encountered 
during startup: Unknown commitlog version 7ERROR [main] 2020-05-07 01:13:26,798 
CassandraDaemon.java:749 - Exception encountered during 
startupjava.lang.IllegalStateException: Unknown commitlog version 7 at 
org.apache.cassandra.db.commitlog.CommitLogDescriptor.getMessagingVersion(CommitLogDescriptor.java:227)
 ~[main/:na] at 
org.apache.cassandra.db.commitlog.CommitLogReader.shouldSkipSegmentId(CommitLogReader.java:276)
 ~[main/:na] at 
org.apache.cassandra.db.commitlog.CommitLogReader.readCommitLogSegment(CommitLogReader.java:220)
 ~[main/:na] at 
org.apache.cassandra.db.commitlog.CommitLogReader.readAllFiles(CommitLogReader.java:132)
 ~[main/:na] at 
org.apache.cassandra.db.commitlog.CommitLogReplayer.replayFiles(CommitLogReplayer.java:137)
 ~[main/:na] at 
org.apache.cassandra.db.commitlog.CommitLog.recoverFiles(CommitLog.java:167) 
~[main/:na] at 
org.apache.cassandra.db.commitlog.CommitLog.recoverSegmentsOnDisk(CommitLog.java:148)
 ~[main/:na] at 
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:324) 
[main/:na] at 
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:620) 
[main/:na] at 
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:732) 
[main/:na]

[jira] [Created] (CASSANDRA-15794) Upgraded C* (4.x) fail to start because of Compact Tables & downgraded C* (3.11.4) fail to start because of commit log

2020-05-06 Thread Zhuqi Jin (Jira)
Zhuqi Jin created CASSANDRA-15794:
-

 Summary: Upgraded C* (4.x) fail to start because of Compact Tables 
& downgraded C* (3.11.4) fail to start because of commit log
 Key: CASSANDRA-15794
 URL: https://issues.apache.org/jira/browse/CASSANDRA-15794
 Project: Cassandra
  Issue Type: Bug
Reporter: Zhuqi Jin


We tried to test upgrading a 3.11.4 C* cluster to 4.x and run into the 
following problems. 
 * We started a single 3.11.4 C* node. 
 * We ran cassandra-stress like this

{code:java}
./cassandra-stress write n = 30 -rate threads = 10 -node  172.17.0.2 {code}
 * We stopped this node, and started a C* node running C* compiled from trunk 
(git commit: e394dc0bb32f612a476269010930c617dd1ed3cb)
 * New C* failed to start with the following error message

{code:java}
ERROR [main] 2020-05-07 00:58:18,503 CassandraDaemon.java:245 - Error while 
loading schema: ERROR [main] 2020-05-07 00:58:18,503 CassandraDaemon.java:245 - 
Error while loading schema: java.lang.IllegalArgumentException: Compact Tables 
are not allowed in Cassandra starting with 4.0 version. Use `ALTER ... DROP 
COMPACT STORAGE` command supplied in 3.x/3.11 Cassandra in order to migrate off 
Compact Storage. at 
org.apache.cassandra.schema.SchemaKeyspace.fetchTable(SchemaKeyspace.java:965) 
at 
org.apache.cassandra.schema.SchemaKeyspace.fetchTables(SchemaKeyspace.java:924) 
at 
org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:883)
 at 
org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:874)
 at 
org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:862)
 at org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:102) at 
org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:91) at 
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:241) at 
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:653) 
at 
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:770)Exception
 (java.lang.IllegalArgumentException) encountered during startup: Compact 
Tables are not allowed in Cassandra starting with 4.0 version. Use `ALTER ... 
DROP COMPACT STORAGE` command supplied in 3.x/3.11 Cassandra in order to 
migrate off Compact Storage.ERROR [main] 2020-05-07 00:58:18,520 
CassandraDaemon.java:792 - Exception encountered during 
startupjava.lang.IllegalArgumentException: Compact Tables are not allowed in 
Cassandra starting with 4.0 version. Use `ALTER ... DROP COMPACT STORAGE` 
command supplied in 3.x/3.11 Cassandra in order to migrate off Compact Storage. 
at 
org.apache.cassandra.schema.SchemaKeyspace.fetchTable(SchemaKeyspace.java:965) 
at 
org.apache.cassandra.schema.SchemaKeyspace.fetchTables(SchemaKeyspace.java:924) 
at 
org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:883)
 at 
org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:874)
 at 
org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:862)
 at org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:102) at 
org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:91) at 
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:241) at 
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:653) 
at 
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:770){code}
 * We stopped the trunk version C* and started the 3.11.4 version C*. 
 * 3.11.4 C* failed to start with the following error messages: 

{code:java}
Exception (java.lang.IllegalStateException) encountered during startup: Unknown 
commitlog version 7Exception (java.lang.IllegalStateException) encountered 
during startup: Unknown commitlog version 7ERROR [main] 2020-05-07 01:13:26,798 
CassandraDaemon.java:749 - Exception encountered during 
startupjava.lang.IllegalStateException: Unknown commitlog version 7 at 
org.apache.cassandra.db.commitlog.CommitLogDescriptor.getMessagingVersion(CommitLogDescriptor.java:227)
 ~[main/:na] at 
org.apache.cassandra.db.commitlog.CommitLogReader.shouldSkipSegmentId(CommitLogReader.java:276)
 ~[main/:na] at 
org.apache.cassandra.db.commitlog.CommitLogReader.readCommitLogSegment(CommitLogReader.java:220)
 ~[main/:na] at 
org.apache.cassandra.db.commitlog.CommitLogReader.readAllFiles(CommitLogReader.java:132)
 ~[main/:na] at 
org.apache.cassandra.db.commitlog.CommitLogReplayer.replayFiles(CommitLogReplayer.java:137)
 ~[main/:na] at 
org.apache.cassandra.db.commitlog.CommitLog.recoverFiles(CommitLog.java:167) 
~[main/:na] at 
org.apache.cassandra.db.commitlog.CommitLog.recoverSegmentsOnDisk(CommitLog.java:148)
 ~[main/:na] at 
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:324) 
[main/:na] at