[ https://issues.apache.org/jira/browse/CASSANDRA-17366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17522978#comment-17522978 ]
Brandon Williams commented on CASSANDRA-17366: ---------------------------------------------- These tests setup and then stop a cluster, subsequently starting it with some combination of seeds, and then nodes in parallel. The problem is that the setup doesn't guarantee it will wait long enough for the cluster to be completely established, though C* is fast enough to do so anyway, _almost_ all of the time. To ensure the setup is complete before shutting down, the nodes should wait for the CQL interface to become available after the initial startup. [This dtest branch|https://github.com/driftx/cassandra-dtest/tree/CASSANDRA-17366] does that, and here's 400 runs on [4.0|https://app.circleci.com/pipelines/github/driftx/cassandra/438/workflows/df01fde1-1ff1-4007-bdc2-a9de8e358a16/jobs/5145] and [trunk|https://app.circleci.com/pipelines/github/driftx/cassandra/439/workflows/c00617e8-904c-44d7-9f4f-de565e4878cf/jobs/5143]. > Fix flaky test - gossip_test.TestGossip > --------------------------------------- > > Key: CASSANDRA-17366 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17366 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Gossip > Reporter: Aleksei Zotov > Assignee: Brandon Williams > Priority: Normal > Fix For: 4.0.x, 4.x > > > We can see many failures for 4.x branch: > test_2dc_parallel_startup_one_seed > ([916|https://ci-cassandra.apache.org/job/Cassandra-trunk/916/testReport/dtest-offheap.gossip_test/TestGossip], > > [920|https://ci-cassandra.apache.org/job/Cassandra-trunk/920/testReport/dtest.gossip_test/TestGossip,]) > test_2dc_parallel_startup > ([929|https://ci-cassandra.apache.org/job/Cassandra-trunk/929/testReport/dtest-novnode.gossip_test/TestGossip], > > [931|https://ci-cassandra.apache.org/job/Cassandra-trunk/931/testReport/dtest.gossip_test/TestGossip], > > [936|https://ci-cassandra.apache.org/job/Cassandra-trunk/936/testReport/dtest-novnode.gossip_test/TestGossip]) > test_2dc_parallel_startup_one_seed > ([916|https://ci-cassandra.apache.org/job/Cassandra-trunk/916/testReport/dtest-offheap.gossip_test/TestGossip], > > [920|https://ci-cassandra.apache.org/job/Cassandra-trunk/920/testReport/dtest.gossip_test/TestGossip/]) > The error is always the same: > {code:java} > Unexpected error found in node logs (see stdout for full details). Errors: > [ERROR [main] 2022-01-26 10:53:12,866 CassandraDaemon.java:900 - Exception > encountered during startup > java.lang.RuntimeException: Didn't receive schemas for all known versions > within the timeout. Use -Dcassandra.skip_schema_check=true to skip this check. > at > org.apache.cassandra.service.StorageService.waitForSchema(StorageService.java:1037) > at > org.apache.cassandra.dht.BootStrapper.allocateTokens(BootStrapper.java:232) > at > org.apache.cassandra.dht.BootStrapper.getBootstrapTokens(BootStrapper.java:180) > at > org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:1089) > at > org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:1043) > at > org.apache.cassandra.service.StorageService.initServer(StorageService.java:821) > at > org.apache.cassandra.service.StorageService.initServer(StorageService.java:751) > at > org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:417) > at > org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:754) > at > org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:878), > ERROR [main] 2022-01-26 10:53:12,866 CassandraDaemon.java:900 - Exception > encountered during startup > java.lang.RuntimeException: Didn't receive schemas for all known versions > within the timeout. Use -Dcassandra.skip_schema_check=true to skip this check. > at > org.apache.cassandra.service.StorageService.waitForSchema(StorageService.java:1037) > at > org.apache.cassandra.dht.BootStrapper.allocateTokens(BootStrapper.java:232) > at > org.apache.cassandra.dht.BootStrapper.getBootstrapTokens(BootStrapper.java:180) > at > org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:1089) > at > org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:1043) > at > org.apache.cassandra.service.StorageService.initServer(StorageService.java:821) > at > org.apache.cassandra.service.StorageService.initServer(StorageService.java:751) > at > org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:417) > at > org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:754) > at > org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:878)] > {code} -- This message was sent by Atlassian Jira (v8.20.1#820001) --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org