I am not making a recommendation for anyone else – just sharing our experience and reasoning. It is why I argued for keeping PropertyFileSnitch in some JIRA that proposed dropping it completely. GPFS is the typical recommendation for production use. Just a hurdle not worth my time at the moment.
From: Alexander Dejanovski <a...@thelastpickle.com> Sent: Wednesday, February 27, 2019 9:22 AM To: user@cassandra.apache.org Subject: Re: [EXTERNAL] Re: Question on changing node IP address It has to be balanced with the dangers related to the PropertyFileSnitch. I've seen such incidents happen twice in the last few months in different places and both times recovery was difficult and hazardous. I still strongly recommend against it. On Wed, Feb 27, 2019 at 3:11 PM Durity, Sean R <sean_r_dur...@homedepot.com<mailto:sean_r_dur...@homedepot.com>> wrote: We use the PropertyFileSnitch precisely because it is the same on every node. If each node has to have a different file (for GPFS) – deployment is more complicated. (And for any automated configuration you would have a list of hosts and DC/rack information to compile anyway) I do put UNKNOWN as the default DC so that any missed node easily appears in its own unused DC. Sean Durity From: Alexander Dejanovski <a...@thelastpickle.com<mailto:a...@thelastpickle.com>> Sent: Wednesday, February 27, 2019 4:43 AM To: user@cassandra.apache.org<mailto:user@cassandra.apache.org> Subject: [EXTERNAL] Re: Question on changing node IP address This snitch is easy to misconfigure. It allows some nodes to have a different view of the cluster if they are configured differently, which can result in data loss (or at least data that is very hard to recover). Also it has a nasty feature that allows to set a default DC/Rack. If one node isn't properly declared in all the files throughout the cluster, it will be seen as part of that "default" DC and then again, it's hard to recover. Be aware that while the GossipingPropertyFileSnitch will not allow changing rack of DC for a node that already bootstrapped, the PropertyFileSnitch will allow to change it without any notice. So a little misconfiguration could merge all nodes from DC1 into DC2, abruptly changing token ownership (and it could very be the case that DC1 thinks it's part of DC2 but DC2 still thinks DC1 is DC1). So again, I think this snitch is dangerous and shouldn't be used. The GossipingPropertyFileSnitch is much more secure and easy to use. Cheers, On Wed, Feb 27, 2019 at 10:13 AM shalom sagges <shalomsag...@gmail.com<mailto:shalomsag...@gmail.com>> wrote: If you're using the PropertyFileSnitch, well... you shouldn't as it's a rather dangerous and tedious snitch to use I inherited Cassandra clusters that use the PropertyFileSnitch. It's been working fine, but you've kinda scared me :-) Why is it dangerous to use? If I decide to change the snitch, is it seamless or is there a specific procedure one must follow? Thanks! On Wed, Feb 27, 2019 at 10:08 AM Alexander Dejanovski <a...@thelastpickle.com<mailto:a...@thelastpickle.com>> wrote: I confirm what Oleksandr said. Just stop Cassandra, change the IP, and restart Cassandra. If you're using the GossipingPropertyFileSnitch, the node will redeclare its new IP through Gossip and that's it. If you're using the PropertyFileSnitch, well... you shouldn't as it's a rather dangerous and tedious snitch to use. But if you are, it'll require to change the file containing all the IP addresses across the cluster. I've been changing IPs on a whole cluster back in 2.1 this way and it went through seamlessly. Cheers, On Wed, Feb 27, 2019 at 8:54 AM Oleksandr Shulgin <oleksandr.shul...@zalando.de<mailto:oleksandr.shul...@zalando.de>> wrote: On Wed, Feb 27, 2019 at 4:15 AM wxn...@zjqunshuo.com<mailto:wxn...@zjqunshuo.com> <wxn...@zjqunshuo.com<mailto:wxn...@zjqunshuo.com>> wrote: >After restart with the new address the server will notice it and log a >warning, but it will keep token ownership as long as it keeps the old host id >(meaning it must use the same data directory as before restart). Based on my understanding, token range is binded to host id. As long as host id doesn't change, everything is ok. Besides data directory, any other thing can lead to host id change? And how host id is caculated? For example, if I upgrade Cassandra binary to a new version, after restart, will host id change? I believe host id is calculated once the new node is initialized and never changes afterwards, even through major upgrades. It is stored in system keyspace in data directory, and is stable across restarts. -- Alex -- ----------------- Alexander Dejanovski France @alexanderdeja Consultant Apache Cassandra Consulting http://www.thelastpickle.com<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.thelastpickle.com_&d=DwMFaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=xojzDh-fJSOl_ZfDMCYIYi4sckWpwqdnKDG5QMx2nUE&s=HwAxm8xI-Bmc8IFmwEK0we9hlvNwUVuj7DGpXuNM8r4&e=> -- ----------------- Alexander Dejanovski France @alexanderdeja Consultant Apache Cassandra Consulting http://www.thelastpickle.com<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.thelastpickle.com_&d=DwMFaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=xojzDh-fJSOl_ZfDMCYIYi4sckWpwqdnKDG5QMx2nUE&s=HwAxm8xI-Bmc8IFmwEK0we9hlvNwUVuj7DGpXuNM8r4&e=> ________________________________ The information in this Internet Email is confidential and may be legally privileged. It is intended solely for the addressee. Access to this Email by anyone else is unauthorized. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful. When addressed to our clients any opinions or advice contained in this Email are subject to the terms and conditions expressed in any applicable governing The Home Depot terms of business or client engagement letter. The Home Depot disclaims all responsibility and liability for the accuracy and content of this attachment and for any damages or losses arising from any inaccuracies, errors, viruses, e.g., worms, trojan horses, etc., or other items of a destructive nature, which may be contained in this attachment and shall not be liable for direct, indirect, consequential or special damages in connection with this e-mail message or its attachment. -- ----------------- Alexander Dejanovski France @alexanderdeja Consultant Apache Cassandra Consulting http://www.thelastpickle.com<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.thelastpickle.com_&d=DwMFaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=Oca8DvijXd-ubvKY9U4yBzzrYELUFspL5jjRVzzbcGw&s=b4GHJdsm3bd3sj27uupBCf_b_r-bU6XEwqkOxe7_fxA&e=> ________________________________ The information in this Internet Email is confidential and may be legally privileged. It is intended solely for the addressee. Access to this Email by anyone else is unauthorized. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful. When addressed to our clients any opinions or advice contained in this Email are subject to the terms and conditions expressed in any applicable governing The Home Depot terms of business or client engagement letter. The Home Depot disclaims all responsibility and liability for the accuracy and content of this attachment and for any damages or losses arising from any inaccuracies, errors, viruses, e.g., worms, trojan horses, etc., or other items of a destructive nature, which may be contained in this attachment and shall not be liable for direct, indirect, consequential or special damages in connection with this e-mail message or its attachment.
