Re: postmortem on 2.2.13 scale out difficulties

2019-06-12 Thread Carl Mueller
we get >> stuck in that somehow. >> >> On Wed, Jun 12, 2019 at 11:45 AM ZAIDI, ASAD A wrote: >> >>> >>> >>> >>> >>> Adding one node at a time – is that successful? >>> >>> Check value of streaming_socket_timeout_in_ms parameter

Re: postmortem on 2.2.13 scale out difficulties

2019-06-12 Thread Carl Mueller
t; >> Have you tried Nodetool bootstrap resume & jvm option i.e. >> JVM_OPTS="$JVM_OPTS -Dcassandra.consistent.rangemovement=false" ? >> >> >> >> >> >> *From:* Carl Mueller [mailto:carl.muel...@smartthings.com.INVALID] >> *

Re: postmortem on 2.2.13 scale out difficulties

2019-06-12 Thread Carl Mueller
; Have you tried Nodetool bootstrap resume & jvm option i.e. > JVM_OPTS="$JVM_OPTS -Dcassandra.consistent.rangemovement=false" ? > > > > > > *From:* Carl Mueller [mailto:carl.muel...@smartthings.com.INVALID] > *Sent:* Wednesday, June 12, 2019 11:35 AM > *To

RE: postmortem on 2.2.13 scale out difficulties

2019-06-12 Thread ZAIDI, ASAD A
: Carl Mueller [mailto:carl.muel...@smartthings.com.INVALID] Sent: Wednesday, June 12, 2019 11:35 AM To: user@cassandra.apache.org Subject: Re: postmortem on 2.2.13 scale out difficulties We only were able to scale out four nodes and then failures started occurring, including multiple instances of

Re: postmortem on 2.2.13 scale out difficulties

2019-06-12 Thread Carl Mueller
We're getting DEBUG [GossipStage:1] 2019-06-12 15:20:07,797 MigrationManager.java:96 - Not pulling schema because versions match or shouldPullSchemaFrom returned false multiple times, as it contacts the nodes. On Wed, Jun 12, 2019 at 11:35 AM Carl Mueller wrote: > We only were able to scale

Re: postmortem on 2.2.13 scale out difficulties

2019-06-12 Thread Carl Mueller
We only were able to scale out four nodes and then failures started occurring, including multiple instances of nodes joining a cluster without streaming. Sigh. On Tue, Jun 11, 2019 at 3:11 PM Carl Mueller wrote: > We had a three-DC (asia-tokyo/europe/us) cassandra 2.2.13 cluster, AWS, > IPV6 >