>> Cassandra stores hints for the lowest of gc_grace_seconds and max_hint_window_in_ms
Was this a tough design decision or just a bug? It is certainly very surprising behavior. Everything that I've read leads me to believe that gc_grace_seconds was only intended to affect the treatment of *expired* data. Thanks, Cody On Mon, Dec 19, 2016 at 8:10 AM, Alain RODRIGUEZ <arodr...@gmail.com> wrote: > Hi, > > >> - Why setting gc_grace_seconds=0 will disable hints for the table? >> >> It was the first time I heard about this as well when Alexander told us > about that. This read might be helpful http://www.uberobert.com/ > cassandra_gc_grace_disables_hinted_handoff/. Also Alexander I know tested > it. > > *tl;dr*: Cassandra stores hints for the lowest of gc_grace_seconds and > max_hint_window_in_ms > > Still I see no reason not to set gc_grace_seconds to 3 hours as a fix / > workaround. Keeping 3 hours of extra data on disk is something you > definitely want to be able to do. > > >> - How can an expired TTL record be deleted by Cassandra without >> tombstoning or compaction? Aren't SSTables immutable files, and expired >> records are removed through compaction? >> >> > This sounds magical to me as well. The only way I am aware of to drop > tombstone without compaction is having an entire "SSTable expired" that > would be soon be evicted, without compactions. TWCS relies on this property > and make a great use of it. Here is Jeff talk about TWCS: > https://www.youtube.com/watch?v=PWtekUWCIaw. I believe he mentioned that. > > >> - If I only use TTL for deletion, do I still need gc_grace_seconds to >> be bigger than 0? >> >> >> - If I only use TTL for deletion, but use updates as well, do I need >> gc_grace_seconds to be bigger than 0? >> >> > Yes, if you care about hints. Anyway, setting gc_grace_seconds to 0 brings > more troubles than solutions in many cases. Use the value of > max_hint_window_in_ms as a minimal gc_grace_seconds (watch out for the time > units in use, do the math ;-) ) > > Here is a blog I wrote a few months ago about tombstones and deletes > http://thelastpickle.com/blog/2016/07/27/about-deletes-and-tombstones.html. > I hope it will give you interesting insight about tombstones, even if you > do not care about all the "deletes" part. About TTLs, see > http://thelastpickle.com/blog/2016/07/27/about-deletes- > and-tombstones.html#tombstones-drop. There is no need for you to repair > within gc_grace_seconds, but given that "Cassandra stores hints for the > lowest of gc_grace_seconds and max_hint_window_in_ms" I would never use a > lower value than 3 hours (default max_hint_window_in_ms) for > gc_grace_seconds, on any table. > > C*heers, > > > 2016-12-19 15:07 GMT+01:00 Shalom Sagges <shal...@liveperson.com>: > >> Thanks for the explanation Matija, but fortunately, that I know. Forgot >> to mention that I'm using a multi DC cluster. >> I'll try to summarize just the questions I have, because my email was >> indeed quite long :-) >> >> >> - Why setting gc_grace_seconds=0 will disable hints for the table? >> - How can an expired TTL record be deleted by Cassandra without >> tombstoning or compaction? Aren't SSTables immutable files, and expired >> records are removed through compaction? >> - If I only use TTL for deletion, do I still need gc_grace_seconds to >> be bigger than 0? >> - If I only use TTL for deletion, but use updates as well, do I need >> gc_grace_seconds to be bigger than 0? >> >> >> Thanks! >> >> >> Shalom Sagges >> DBA >> T: +972-74-700-4035 <+972%2074-700-4035> >> <http://www.linkedin.com/company/164748> <http://twitter.com/liveperson> >> <http://www.facebook.com/LivePersonInc> We Create Meaningful Connections >> >> >> >> On Mon, Dec 19, 2016 at 2:39 PM, Matija Gobec <matija0...@gmail.com> >> wrote: >> >>> Hi, >>> >>> gc_grace_seconds is used to maintain data consistency in some failure >>> scenarios. When manually deleting data that action creates tombstones which >>> are kept for that defined period before being compacted. If one of the >>> replica nodes is down while deleting data and it gets back up after the >>> gc_grace_seconds defined period your previously delete data will reappear >>> (ghost data). As it is stated in datastax documentation on a single node >>> you can set gc_grace_seconds to 0 and you can do the same for tables that >>> contain only data with TTL. In the mentioned failure scenario your downed >>> node will have data with TTL information and no data inconsistency will >>> happen. >>> >>> On Mon, Dec 19, 2016 at 1:00 PM, Shalom Sagges <shal...@liveperson.com> >>> wrote: >>> >>>> Hi Everyone, >>>> >>>> I was reading a blog on TWCS by Alex Dejanovski from The Last Pickle ( >>>> http://thelastpickle.com/blog/2016/12/08/TWCS-part1.html) >>>> >>>> When I got to the comments section, I didn't understand why setting >>>> gc_grace_seconds to 0 will disable hints for the associated table: >>>> *"It is a very good point that gc_grace_seconds shouldn't be lowered >>>> too much as its impact on hinted handoff is not a well known fact, and >>>> using a value of 0 will purely disable hints on the table."* >>>> >>>> When I tried to read some more about deletes and TTLs, I got to a >>>> Datastax documentation https://docs.datastax.com/en/cassandra/3.0/cas >>>> sandra/dml/dmlAboutDeletes.html >>>> stating the following: >>>> >>>> *Cassandra allows you to set a default_time_to_live property for an >>>> entire table. Columns and rows marked with regular TTLs are processed as >>>> described above; but when a record exceeds the table-level TTL, Cassandra >>>> deletes it immediately, without tombstoning or compaction.* >>>> >>>> Which got me a bit more confused. >>>> So I hope someone can shed some light on some questions I've got: >>>> >>>> >>>> - Why setting gc_grace_seconds=0 will disable hints for the table? >>>> - How can an expired TTL record be deleted by Cassandra without >>>> tombstoning or compaction? Aren't SSTables immutable files, and expired >>>> records are removed through compaction? >>>> - If I only use TTL for deletion, do I still need gc_grace_seconds >>>> to be bigger than 0? >>>> - If I only use TTL for deletion, but use updates as well, do I >>>> need gc_grace_seconds to be bigger than 0? >>>> >>>> >>>> Sorry for all those questions, I'm just really confused from all the >>>> TTL/tombstones subject (still a newbie). >>>> >>>> Thanks a lot! >>>> >>>> >>>> Shalom Sagges >>>> DBA >>>> T: +972-74-700-4035 <+972%2074-700-4035> >>>> <http://www.linkedin.com/company/164748> >>>> <http://twitter.com/liveperson> <http://www.facebook.com/LivePersonInc> We >>>> Create Meaningful Connections >>>> >>>> >>>> >>>> This message may contain confidential and/or privileged information. >>>> If you are not the addressee or authorized to receive this on behalf of >>>> the addressee you must not use, copy, disclose or take action based on this >>>> message or any information herein. >>>> If you have received this message in error, please advise the sender >>>> immediately by reply email and delete this message. Thank you. >>>> >>> >>> >> >> This message may contain confidential and/or privileged information. >> If you are not the addressee or authorized to receive this on behalf of >> the addressee you must not use, copy, disclose or take action based on this >> message or any information herein. >> If you have received this message in error, please advise the sender >> immediately by reply email and delete this message. Thank you. >> > >
