I’m not parsing this - did the lower gcgs help or not ? Seeing the table histograms is the next step if this is still a problem
The table level TTL doesn’t matter if you set a TTL on each insert -- Jeff Jirsa > On Feb 23, 2019, at 4:37 PM, Rahul Reddy <rahulreddy1...@gmail.com> wrote: > > Thanks Jeff, > > I'm having gcgs set to 10 mins and changed the table ttl also to 5 hours > compared to insert ttl to 4 hours . Tracing on doesn't show any tombstone > scans for the reads. And also log doesn't show tombstone scan alerts. Has > the reads are happening 5-8k reads per node during the peak hours it shows 1M > tombstone scans count per read. > >> On Fri, Feb 22, 2019, 11:46 AM Jeff Jirsa <jji...@gmail.com> wrote: >> If all of your data is TTL’d and you never explicitly delete a cell without >> using s TTL, you can probably drop your GCGS to 1 hour (or less). >> >> Which compaction strategy are you using? You need a way to clear out those >> tombstones. There exist tombstone compaction sub properties that can help >> encourage compaction to grab sstables just because they’re full of >> tombstones which will probably help you. >> >> >> -- >> Jeff Jirsa >> >> >>> On Feb 22, 2019, at 8:37 AM, Kenneth Brotman <kenbrot...@yahoo.com.invalid> >>> wrote: >>> >>> Can we see the histogram? Why wouldn’t you at times have that many >>> tombstones? Makes sense. >>> >>> >>> >>> Kenneth Brotman >>> >>> >>> >>> From: Rahul Reddy [mailto:rahulreddy1...@gmail.com] >>> Sent: Thursday, February 21, 2019 7:06 AM >>> To: user@cassandra.apache.org >>> Subject: Tombstones in memtable >>> >>> >>> >>> We have small table records are about 5k . >>> >>> All the inserts comes as 4hr ttl and we have table level ttl 1 day and gc >>> grace seconds has 3 hours. We do 5k reads a second during peak load During >>> the peak load seeing Alerts for tomstone scanned histogram reaching million. >>> >>> Cassandra version 3.11.1. Please let me know how can this tombstone scan >>> can be avoided in memtable
