Re: High latency on 5 node Cassandra Cluster

2014-06-05 Thread Arup Chakrabarti
Yes, I meant RF: 2,2,1. We have the following in our network topo config: # cass08: us-west-1c private.ip=DC-A:1 public.ip=DC-A:1 # cass09: us-west-1b private.ip=DC-A:2 public.ip=DC-A:2 # cass10: Linode-Fremont-CA public.ip=DC-C:1 # cass11: us-west-2c private.ip=DC-B:1 public.ip=DC-B:1

Re: High latency on 5 node Cassandra Cluster

2014-06-04 Thread Robert Coli
On Wed, Jun 4, 2014 at 12:12 AM, Arup Chakrabarti wrote: > Size: 5 nodes (2 in AWS US-West-1, 2 in AWS US-West-2, 1 in Linode Fremont) > Replication Factor: 5 > You're operating with a single-DC strategy across multiple data centers? If so, I'm surprised you get sane latency ever. (Or do you me

Re: High latency on 5 node Cassandra Cluster

2014-06-04 Thread Nate McCall
That is a pretty old version of Cassandra at this point. If you are using counters anywhere, you are probably seeing https://issues.apache.org/jira/browse/CASSANDRA-4578 which only shows up after you hit some arbitrary traffic threshold. If you don't want to upgrade (you really should), there was

Re: High latency on 5 node Cassandra Cluster

2014-06-04 Thread Laing, Michael
I would first check to see if there was a time synchronization issue among nodes that triggered and/or perpetuated the event. ml On Wed, Jun 4, 2014 at 3:12 AM, Arup Chakrabarti wrote: > Hello. We had some major latency problems yesterday with our 5 node > cassandra cluster. Wanted to get some

High latency on 5 node Cassandra Cluster

2014-06-04 Thread Arup Chakrabarti
Hello. We had some major latency problems yesterday with our 5 node cassandra cluster. Wanted to get some feedback on where we could start to look to figure out what was causing the issue. If there is more info I should provide, please let me know. Here are the basics of the cluster: Clients: Hect