>From http://www.uberobert.com/cassandra_gc_grace_disables_hinted_handoff/
This is just a quick FYI post as I don't see this documented on the web > elsewhere. As of now in all versions of Cassandra a gc_grace_seconds setting > of 0 will disable hinted handoff. Basically to avoid an edge case that > could cause data to reappear in a cluster (Detailed in Jira CASSANDRA-5314 > <https://issues.apache.org/jira/browse/CASSANDRA-5314>) hints are stored > with a TTL of gc_grace_seconds for the keyspace in question. A > gc_grace_seconds setting of 0 will cause hints to TTL instantly and they > will never be streamed off when a node comes back up. I did not read the ticket yet, but it might bring some enlightening as well regarding your question Cody, C*heers, ----------------------- Alain Rodriguez - @arodream - al...@thelastpickle.com France The Last Pickle - Apache Cassandra Consulting http://www.thelastpickle.com 2016-12-19 16:39 GMT+01:00 Cody Yancey <yan...@uber.com>: > >> Cassandra stores hints for the lowest of gc_grace_seconds and > max_hint_window_in_ms > > Was this a tough design decision or just a bug? It is certainly very > surprising behavior. Everything that I've read leads me to believe that > gc_grace_seconds was only intended to affect the treatment of *expired* > data. > > Thanks, > Cody > > On Mon, Dec 19, 2016 at 8:10 AM, Alain RODRIGUEZ <arodr...@gmail.com> > wrote: > >> Hi, >> >> >>> - Why setting gc_grace_seconds=0 will disable hints for the table? >>> >>> It was the first time I heard about this as well when Alexander told us >> about that. This read might be helpful http://www.uberobert.com/cassa >> ndra_gc_grace_disables_hinted_handoff/. Also Alexander I know tested it. >> >> *tl;dr*: Cassandra stores hints for the lowest of gc_grace_seconds and >> max_hint_window_in_ms >> >> Still I see no reason not to set gc_grace_seconds to 3 hours as a fix / >> workaround. Keeping 3 hours of extra data on disk is something you >> definitely want to be able to do. >> >> >>> - How can an expired TTL record be deleted by Cassandra without >>> tombstoning or compaction? Aren't SSTables immutable files, and expired >>> records are removed through compaction? >>> >>> >> This sounds magical to me as well. The only way I am aware of to drop >> tombstone without compaction is having an entire "SSTable expired" that >> would be soon be evicted, without compactions. TWCS relies on this property >> and make a great use of it. Here is Jeff talk about TWCS: >> https://www.youtube.com/watch?v=PWtekUWCIaw. I believe he mentioned that. >> >> >>> - If I only use TTL for deletion, do I still need gc_grace_seconds >>> to be bigger than 0? >>> >>> >>> - If I only use TTL for deletion, but use updates as well, do I need >>> gc_grace_seconds to be bigger than 0? >>> >>> >> Yes, if you care about hints. Anyway, setting gc_grace_seconds to 0 >> brings more troubles than solutions in many cases. Use the value of >> max_hint_window_in_ms as a minimal gc_grace_seconds (watch out for the time >> units in use, do the math ;-) ) >> >> Here is a blog I wrote a few months ago about tombstones and deletes >> http://thelastpickle.com/blog/2016/07/27/about-deletes-and-t >> ombstones.html. I hope it will give you interesting insight about >> tombstones, even if you do not care about all the "deletes" part. About >> TTLs, see http://thelastpickle.com/blog/2016/07/27/about-deletes-a >> nd-tombstones.html#tombstones-drop. There is no need for you to repair >> within gc_grace_seconds, but given that "Cassandra stores hints for the >> lowest of gc_grace_seconds and max_hint_window_in_ms" I would never use a >> lower value than 3 hours (default max_hint_window_in_ms) for >> gc_grace_seconds, on any table. >> >> C*heers, >> >> >> 2016-12-19 15:07 GMT+01:00 Shalom Sagges <shal...@liveperson.com>: >> >>> Thanks for the explanation Matija, but fortunately, that I know. Forgot >>> to mention that I'm using a multi DC cluster. >>> I'll try to summarize just the questions I have, because my email was >>> indeed quite long :-) >>> >>> >>> - Why setting gc_grace_seconds=0 will disable hints for the table? >>> - How can an expired TTL record be deleted by Cassandra without >>> tombstoning or compaction? Aren't SSTables immutable files, and expired >>> records are removed through compaction? >>> - If I only use TTL for deletion, do I still need gc_grace_seconds >>> to be bigger than 0? >>> - If I only use TTL for deletion, but use updates as well, do I need >>> gc_grace_seconds to be bigger than 0? >>> >>> >>> Thanks! >>> >>> >>> Shalom Sagges >>> DBA >>> T: +972-74-700-4035 <+972%2074-700-4035> >>> <http://www.linkedin.com/company/164748> <http://twitter.com/liveperson> >>> <http://www.facebook.com/LivePersonInc> We Create Meaningful Connections >>> >>> >>> >>> On Mon, Dec 19, 2016 at 2:39 PM, Matija Gobec <matija0...@gmail.com> >>> wrote: >>> >>>> Hi, >>>> >>>> gc_grace_seconds is used to maintain data consistency in some failure >>>> scenarios. When manually deleting data that action creates tombstones which >>>> are kept for that defined period before being compacted. If one of the >>>> replica nodes is down while deleting data and it gets back up after the >>>> gc_grace_seconds defined period your previously delete data will reappear >>>> (ghost data). As it is stated in datastax documentation on a single node >>>> you can set gc_grace_seconds to 0 and you can do the same for tables that >>>> contain only data with TTL. In the mentioned failure scenario your downed >>>> node will have data with TTL information and no data inconsistency will >>>> happen. >>>> >>>> On Mon, Dec 19, 2016 at 1:00 PM, Shalom Sagges <shal...@liveperson.com> >>>> wrote: >>>> >>>>> Hi Everyone, >>>>> >>>>> I was reading a blog on TWCS by Alex Dejanovski from The Last Pickle ( >>>>> http://thelastpickle.com/blog/2016/12/08/TWCS-part1.html) >>>>> >>>>> When I got to the comments section, I didn't understand why setting >>>>> gc_grace_seconds to 0 will disable hints for the associated table: >>>>> *"It is a very good point that gc_grace_seconds shouldn't be lowered >>>>> too much as its impact on hinted handoff is not a well known fact, and >>>>> using a value of 0 will purely disable hints on the table."* >>>>> >>>>> When I tried to read some more about deletes and TTLs, I got to a >>>>> Datastax documentation https://docs.datastax.com/en/cassandra/3.0/cas >>>>> sandra/dml/dmlAboutDeletes.html >>>>> stating the following: >>>>> >>>>> *Cassandra allows you to set a default_time_to_live property for an >>>>> entire table. Columns and rows marked with regular TTLs are processed as >>>>> described above; but when a record exceeds the table-level TTL, Cassandra >>>>> deletes it immediately, without tombstoning or compaction.* >>>>> >>>>> Which got me a bit more confused. >>>>> So I hope someone can shed some light on some questions I've got: >>>>> >>>>> >>>>> - Why setting gc_grace_seconds=0 will disable hints for the table? >>>>> - How can an expired TTL record be deleted by Cassandra without >>>>> tombstoning or compaction? Aren't SSTables immutable files, and expired >>>>> records are removed through compaction? >>>>> - If I only use TTL for deletion, do I still need gc_grace_seconds >>>>> to be bigger than 0? >>>>> - If I only use TTL for deletion, but use updates as well, do I >>>>> need gc_grace_seconds to be bigger than 0? >>>>> >>>>> >>>>> Sorry for all those questions, I'm just really confused from all the >>>>> TTL/tombstones subject (still a newbie). >>>>> >>>>> Thanks a lot! >>>>> >>>>> >>>>> Shalom Sagges >>>>> DBA >>>>> T: +972-74-700-4035 <+972%2074-700-4035> >>>>> <http://www.linkedin.com/company/164748> >>>>> <http://twitter.com/liveperson> >>>>> <http://www.facebook.com/LivePersonInc> We Create Meaningful >>>>> Connections >>>>> >>>>> >>>>> >>>>> This message may contain confidential and/or privileged information. >>>>> If you are not the addressee or authorized to receive this on behalf >>>>> of the addressee you must not use, copy, disclose or take action based on >>>>> this message or any information herein. >>>>> If you have received this message in error, please advise the sender >>>>> immediately by reply email and delete this message. Thank you. >>>>> >>>> >>>> >>> >>> This message may contain confidential and/or privileged information. >>> If you are not the addressee or authorized to receive this on behalf of >>> the addressee you must not use, copy, disclose or take action based on this >>> message or any information herein. >>> If you have received this message in error, please advise the sender >>> immediately by reply email and delete this message. Thank you. >>> >> >> >
