[ https://issues.apache.org/jira/browse/CASSANDRA-9694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14611820#comment-14611820 ]
Sam Tunnicliffe commented on CASSANDRA-9694: -------------------------------------------- The new {{system_auth}} tables - {{roles}}, {{role_members}}, {{role_permissions}} & {{resource_role_permissions_index}} - are created on each node as it is upgraded. When a 2.2 node comes up, if it detects the old tables are present it will attempt the conversion to the new tables. This conversion will necessarily fail until enough nodes have been upgraded. You'll see log messages to this effect on the upgraded nodes. e.g.: {noformat} INFO [OptionalTasks:1] 2015-07-02 12:15:21,510 CassandraRoleManager.java:380 - Converting legacy users INFO [OptionalTasks:1] 2015-07-02 12:15:23,539 CassandraRoleManager.java:413 - Unable to complete conversion of legacy auth data (perhaps not enough nodes are upgraded yet). Conversion should not be considered complete INFO [OptionalTasks:1] 2015-07-02 12:15:23,539 CassandraAuthorizer.java:396 - Converting legacy permissions data INFO [OptionalTasks:1] 2015-07-02 12:15:25,544 CassandraAuthorizer.java:440 - Unable to complete conversion of legacy permissions data (perhaps not enough nodes are upgraded yet). Conversion should not be considered complete {noformat} While the cluster is in the mixed state, authentication & authorization will continue to use the old tables, even on the upgraded nodes. Once all nodes have been upgraded & the data conversion completed, the legacy system_auth tables {{users}}, {{credentials}} & {{permissions}} should be dropped. For safety reasons this is not done automatically, so an operator with su privileges needs to do this via cqlsh. Once those tables are removed, auth will automatically begin using the new tables without any further intervention. You can verify that the migration has happened correctly from the system log on upgraded nodes. Once enough 2.2 nodes are available, the messages will change from those above to: {noformat} INFO [OptionalTasks:1] 2015-07-02 12:23:05,222 CassandraRoleManager.java:380 - Converting legacy users INFO [OptionalTasks:1] 2015-07-02 12:23:05,252 CassandraRoleManager.java:390 - Completed conversion of legacy users INFO [OptionalTasks:1] 2015-07-02 12:23:05,252 CassandraRoleManager.java:395 - Migrating legacy credentials data to new system table INFO [OptionalTasks:1] 2015-07-02 12:23:05,265 CassandraRoleManager.java:408 - Completed conversion of legacy credentials INFO [OptionalTasks:1] 2015-07-02 12:23:05,265 CassandraAuthorizer.java:396 - Converting legacy permissions data INFO [OptionalTasks:1] 2015-07-02 12:23:05,274 CassandraAuthorizer.java:435 - Completed conversion of legacy permissions {noformat} This isn't quite as clear as it could be in NEWS.txt, so I'm attaching a patch to clarify Finally, on 2.2.0-rc1 you'll notice a delay of ~10s logging into cqlsh when the cluster is in a mixed state. This is due to the bundled python driver attempting to wait for a schema agreement that will never come. It is resolved in 2.2.0-rc2 by virtue of the bundled driver incorporating [PYTHON-303|https://datastax-oss.atlassian.net/browse/PYTHON-303] > system_auth not upgraded > ------------------------ > > Key: CASSANDRA-9694 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9694 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: Windows-7-32 bit, 3.2GB RAM, Java 1.7.0_55 > Reporter: Andreas Schnitzerling > Assignee: Sam Tunnicliffe > Attachments: system_exception.log > > > After upgrading Authorization-Exceptions occur. I checked the system_auth > keyspace and have seen, that tables users, credentials and permissions were > not upgraded automatically. I upgraded them (I needed 2 times per table > because of CASSANDRA-9566). After upgrading the system_auth tables I could > login via cql using different users. > {code:title=system.log} > WARN [Thrift:14] 2015-07-01 11:38:57,748 CassandraAuthorizer.java:91 - > CassandraAuthorizer failed to authorize #<User updateprog> for <keyspace > logdata> > ERROR [Thrift:14] 2015-07-01 11:41:26,210 CustomTThreadPoolServer.java:223 - > Error occurred during processing of message. > com.google.common.util.concurrent.UncheckedExecutionException: > java.lang.RuntimeException: > org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - > received only 0 responses. > at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2201) > ~[guava-16.0.jar:na] > at com.google.common.cache.LocalCache.get(LocalCache.java:3934) > ~[guava-16.0.jar:na] > at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3938) > ~[guava-16.0.jar:na] > at > com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4821) > ~[guava-16.0.jar:na] > at > org.apache.cassandra.auth.PermissionsCache.getPermissions(PermissionsCache.java:72) > ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1] > at > org.apache.cassandra.auth.AuthenticatedUser.getPermissions(AuthenticatedUser.java:104) > ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1] > at > org.apache.cassandra.service.ClientState.authorize(ClientState.java:362) > ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1] > at > org.apache.cassandra.service.ClientState.checkPermissionOnResourceChain(ClientState.java:295) > ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1] > at > org.apache.cassandra.service.ClientState.ensureHasPermission(ClientState.java:272) > ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1] > at > org.apache.cassandra.service.ClientState.hasAccess(ClientState.java:259) > ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1] > at > org.apache.cassandra.service.ClientState.hasColumnFamilyAccess(ClientState.java:243) > ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1] > at > org.apache.cassandra.cql3.statements.SelectStatement.checkAccess(SelectStatement.java:143) > ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1] > at > org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:222) > ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1] > at > org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:256) > ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1] > at > org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:241) > ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1] > at > org.apache.cassandra.thrift.CassandraServer.execute_cql3_query(CassandraServer.java:1891) > ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1] > at > org.apache.cassandra.thrift.Cassandra$Processor$execute_cql3_query.getResult(Cassandra.java:4588) > ~[apache-cassandra-thrift-2.2.0-rc1.jar:2.2.0-rc1] > at > org.apache.cassandra.thrift.Cassandra$Processor$execute_cql3_query.getResult(Cassandra.java:4572) > ~[apache-cassandra-thrift-2.2.0-rc1.jar:2.2.0-rc1] > at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) > ~[libthrift-0.9.2.jar:0.9.2] > at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) > ~[libthrift-0.9.2.jar:0.9.2] > at > org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:204) > ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1] > at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) > [na:1.7.0_55] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) > [na:1.7.0_55] > at java.lang.Thread.run(Unknown Source) [na:1.7.0_55] > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)