Re: Node Failure Scenario

2017-11-15 Thread Anshu Vajpayee
Thank you Jonathan and all. On Tue, Nov 14, 2017 at 10:53 PM, Jonathan Haddad wrote: > Anthony’s suggestions using replace_address_first_boot lets you avoid that > requirement, and it’s specifically why it was added in 2.2. > On Tue, Nov 14, 2017 at 1:02 AM Anshu Vajpayee > wrote: > >> ​Thanks

Re: Node Failure Scenario

2017-11-14 Thread Jonathan Haddad
Anthony’s suggestions using replace_address_first_boot lets you avoid that requirement, and it’s specifically why it was added in 2.2. On Tue, Nov 14, 2017 at 1:02 AM Anshu Vajpayee wrote: > ​Thanks guys , > > I thikn better to pass replace_address on command line rather than update > the cassnd

Re: Node Failure Scenario

2017-11-14 Thread Anshu Vajpayee
​Thanks guys , I thikn better to pass replace_address on command line rather than update the cassndra-env file so that there would not be requirement to remove it later. ​ On Tue, Nov 14, 2017 at 6:32 AM, Anthony Grasso wrote: > Hi Anshu, > > To add to Erick's comment, remember to remove the

Re: Node Failure Scenario

2017-11-13 Thread Anthony Grasso
Hi Anshu, To add to Erick's comment, remember to remove the *replace_address* method from the *cassandra-env.sh* file once the node has rejoined successfully. The node will fail the next restart otherwise. Alternatively, use the *replace_address_first_boot* method which works exactly the same way

Re: Node Failure Scenario

2017-11-12 Thread Erick Ramirez
Use the replace_address method with its own IP address. Make sure you delete the contents of the following directories: - data/ - commitlog/ - saved_caches/ Forget rejoining with repair -- it will just cause more problems. Cheers! On Mon, Nov 13, 2017 at 2:54 PM, Anshu Vajpayee wrote: > Hi All

Re: Node failure

2017-10-06 Thread Jon Haddad
I’ve had a few use cases for downgrading consistency over the years. If you’re showing a customer dashboard w/ some Ad summary data, it’s great to be right, but showing a number that’s close is better than not being up. > On Oct 6, 2017, at 1:32 PM, Jeff Jirsa wrote: > > I think it was Brando

Re: Node failure

2017-10-06 Thread Jeff Jirsa
I think it was Brandon that used to make a pretty compelling argument that downgrading consistency on writes was always wrong, because if you can tolerate the lower consistency, you should just use the lower consistency from the start (because cassandra is still going to send the write to all repli

Re: Node failure

2017-10-06 Thread Jim Witschey
> Modern client drivers also have ways to “downgrade” the CL of requests, in > case they fail. E.g. for the Java driver: > http://docs.datastax.com/en/latest-java-driver-api/com/datastax/driver/core/policies/DowngradingConsistencyRetryPolicy.html Quick note from a driver dev's perspective: Mark,

RE: Node failure

2017-10-06 Thread Mark Furlong
I’ll check to see what our app is using. Thanks Mark 801-705-7115 office From: Steinmaurer, Thomas [mailto:thomas.steinmau...@dynatrace.com] Sent: Friday, October 6, 2017 12:25 PM To: user@cassandra.apache.org Subject: RE: Node failure QUORUM should succeed with a RF=3 and 2 of 3 nodes

RE: Node failure

2017-10-06 Thread Steinmaurer, Thomas
/DowngradingConsistencyRetryPolicy.html Thomas From: Mark Furlong [mailto:mfurl...@ancestry.com] Sent: Freitag, 06. Oktober 2017 19:43 To: user@cassandra.apache.org Subject: RE: Node failure Thanks for the detail. I’ll have to remove and then add one back in. It’s my consistency levels that may bite me in the interim. Thanks

RE: Node failure

2017-10-06 Thread Mark Furlong
We are using quorum on our reads and writes. Thanks Mark 801-705-7115 office From: Jeff Jirsa [mailto:jji...@gmail.com] Sent: Friday, October 6, 2017 11:30 AM To: cassandra Subject: Re: Node failure If you write with CL:ANY, CL:ONE (or LOCAL_ONE), and one node fails, you may lose data that

RE: Node failure

2017-10-06 Thread Mark Furlong
Thanks for the detail. I’ll have to remove and then add one back in. It’s my consistency levels that may bite me in the interim. Thanks Mark 801-705-7115 office From: Jeff Jirsa [mailto:jji...@gmail.com] Sent: Friday, October 6, 2017 11:29 AM To: cassandra Subject: Re: Node failure There

Re: Node failure

2017-10-06 Thread Jeff Jirsa
ld be aware of? > > > > *Thanks* > > *Mark* > > *801-705-7115 <(801)%20705-7115> office* > > > > *From:* Akshit Jain [mailto:akshit13...@iiitd.ac.in] > *Sent:* Friday, October 6, 2017 11:25 AM > *To:* user@cassandra.apache.org > *Subject:* Re: Node f

RE: Node failure

2017-10-06 Thread Mark Furlong
The only time I’ll have a problem is if I have a do a read all or write all. Any other gotchas I should be aware of? Thanks Mark 801-705-7115 office From: Akshit Jain [mailto:akshit13...@iiitd.ac.in] Sent: Friday, October 6, 2017 11:25 AM To: user@cassandra.apache.org Subject: Re: Node failure

Re: Node failure

2017-10-06 Thread Akshit Jain
You replace it with a new node and bootstraping happens.The new node receives data from other two nodes. Rest depends on the scenerio u are asking for. Regards Akshit Jain B-Tech,2013124 9891724697 On Fri, Oct 6, 2017 at 10:50 PM, Mark Furlong wrote: > What happens when I have a 3 node cluster

Re: Node failure

2017-10-06 Thread Jeff Jirsa
There's a lot to talk about here, what's your exact question? - You can either remove it from the cluster or replace it. You typically remove it if it'll never be replaced, but in RF=3 with 3 nodes, you probably need to replace it. To replace, you'll start a new server with -Dcassandra.replace_ad

RE: Node failure Due To Very high GC pause time

2017-07-13 Thread Durity, Sean R
distributed across all of your cluster. And you want to delete whole partitions, if at all possible. (Or at least a reasonable number of deletes within a partition.) Sean Durity From: Karthick V [mailto:karthick...@zohocorp.com] Sent: Monday, July 03, 2017 12:47 PM To: user Subject: Re: Node failure Due

RE: Node failure Due To Very high GC pause time

2017-07-03 Thread ZAIDI, ASAD A
your tables with [tombstones], A quick [grep –i tombstone /path/to/system.log] command would tell you what objects are suffering with tombstones! From: Karthick V [mailto:karthick...@zohocorp.com] Sent: Monday, July 03, 2017 11:47 AM To: user Subject: Re: Node failure Due To Very high GC pa

Re: Node failure Due To Very high GC pause time

2017-07-03 Thread Karthick V
Hi Bryan, Thanks for your quick response. We have already tuned our memory and GC based on our hardware specification and it was working fine until yesterday, i.e before facing the below specified delete request. As you specified we will once again look into our GC & memory confi

Re: Node failure Due To Very high GC pause time

2017-07-03 Thread Bryan Cheng
This is a very antagonistic use case for Cassandra :P I assume you're familiar with Cassandra and deletes? (eg. http://thelastpickle.com/blog/2016/07/27/about-deletes-and-tombstones.html, http://docs.datastax.com/en/cassandra/2.1/cassandra/dml/dml_about_deletes_c.html ) That being said, are you gi

Re: node failure, and automatic decommission (or removetoken)

2011-03-01 Thread Mimi Aluminium
It helps, Thanks a lot, miriam On Mon, Feb 28, 2011 at 9:50 PM, Aaron Morton wrote: > I thought there was more to it. > > The steps for move or removing nodes are outlined on the operations page > wiki as you probably know. > > What approach are you considering to rebalancing the token distribut

Re: node failure, and automatic decommission (or removetoken)

2011-02-28 Thread Aaron Morton
I thought there was more to it. The steps for move or removing nodes are outlined on the operations page wiki as you probably know. What approach are you considering to rebalancing the token distribution when removing a node? E.g. If you have 5 nodes and remove 1 the best long term solution is

Re: node failure, and automatic decommission (or removetoken)

2011-02-28 Thread Mimi Aluminium
Aaron, Thanks a lot, Actually I meant a larger number of nodes than 3 and replication factor of 3. We are looking on a system that may shrink due to permanent failures, and then automatically detects the failure and stream its range to other nodes in the cluster to have again 3 replicas. I understn

Re: node failure, and automatic decommission (or removetoken)

2011-02-28 Thread aaron morton
AFAIK the general assumption is that you will want to repair the node manually, within the GCGraceSeconds period. If this cannot be done then nodetool decomission and removetoken are the recommended approach. In your example though, with 3 nodes and an RF of 3 your cluster can sustain a single