[ https://issues.apache.org/jira/browse/CASSANDRA-18845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17766917#comment-17766917 ]
Stefan Miklosovic commented on CASSANDRA-18845: ----------------------------------------------- I noticed now this in your comment above: _This is going to be very difficult todo. dtests setup clusters on loopback addresses and waitToSettle code path has a guard against it if using a loopback address. Also, the problems mostly become apparent with large clusters._ This is really true (1) Gossip.waitToSettle is called only in case it is not on loopback. Since our dtests are all on loopback (right?) I do not think that code was ever invoked during dtests so its revert was not necessary. _If I redo the patch and remove the changes to ECHO and show those tests do not have regression would this allow the ticket to move forward?_ I think that is reasonable, wdyt, [~brandon.williams]? I think that what was unfortunate was that we mixed flood of echos solution / change with waiting for at least one node to be up. I think that the waiting for at least 1 node can go in and we will focus on the echos separately. (1) https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/CassandraDaemon.java#L400-L401 (2) https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/dht/BootStrapper.java#L213-L214 (3) https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/dht/BootStrapper.java#L235-L236 (4) https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/schema/MigrationCoordinator.java#L696-L697 > Waiting for gossip to settle on live endpoints > ---------------------------------------------- > > Key: CASSANDRA-18845 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18845 > Project: Cassandra > Issue Type: Improvement > Reporter: Cameron Zemek > Priority: Normal > Attachments: delay.log, example.log, > image-2023-09-14-11-16-23-020.png, test1.log, test2.log, test3.log > > > This is a follow up to CASSANDRA-18543 > Although that ticket added ability to set cassandra.gossip_settle_min_wait_ms > this is tedious and error prone. On a node just observed a 79 second gap > between waiting for gossip and the first echo response to indicate a node is > UP. > The problem being that do not want to start Native Transport until gossip > settles otherwise queries can fail consistency such as LOCAL_QUORUM as it > thinks the replicas are still in DOWN state. > Instead of having to set gossip_settle_min_wait_ms I am proposing that > (outside single node cluster) wait for UP message from another node before > considering gossip as settled. Eg. > {code:java} > if (currentSize == epSize && currentLive == liveSize && liveSize > > 1) > { > logger.debug("Gossip looks settled."); > numOkay++; > } {code} -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org