[ https://issues.apache.org/jira/browse/CASSANDRA-6127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13804459#comment-13804459 ]
Quentin Conner commented on CASSANDRA-6127: ------------------------------------------- First results with workaround patch #2. No load. No data. Only system keyspace and Gossip on a 256 node m1.medium cluster in EC2. Nodes started in rapid succession. *phi=8, variable gossip_period* 1154 flaps for 1 sec 685 flaps for 2 sec 146 flaps for 3 sec 88 flaps for 4 sec 70 flaps for 5 sec 100 flaps for 10 sec *phi=12* 1289 flaps for 1 sec 77 flaps for 2 sec 6 flaps for 3 sec 1 flaps for 4 sec 3 flaps for 5 sec 1 flaps for 6 sec 0 flaps for 8 sec 1 flaps for 10 sec > vnodes don't scale to hundreds of nodes > --------------------------------------- > > Key: CASSANDRA-6127 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6127 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: Any cluster that has vnodes and consists of hundreds of > physical nodes. > Reporter: Tupshin Harper > Assignee: Jonathan Ellis > Attachments: 6000vnodes.patch, AdjustableGossipPeriod.patch, > delayEstimatorUntilStatisticallyValid.patch > > > There are a lot of gossip-related issues related to very wide clusters that > also have vnodes enabled. Let's use this ticket as a master in case there are > sub-tickets. > The most obvious symptom I've seen is with 1000 nodes in EC2 with m1.xlarge > instances. Each node configured with 32 vnodes. > Without vnodes, cluster spins up fine and is ready to handle requests within > 30 minutes or less. > With vnodes, nodes are reporting constant up/down flapping messages with no > external load on the cluster. After a couple of hours, they were still > flapping, had very high cpu load, and the cluster never looked like it was > going to stabilize or be useful for traffic. -- This message was sent by Atlassian JIRA (v6.1#6144)