Re: TWCS not deleting expired sstables
I would start here: http://thelastpickle.com/blog/2016/12/08/TWCS-part1.html Specifically the “Hints and repairs” and “Timestamp overlap” sections might be of use. -B > On Jan 25, 2018, at 11:05 AM, Thakrar, Jayesh > wrote: > > Wondering if I can get some pointers to what's happening here and why > sstables that I think should be expired are not being dropped. > > Here's the table's compaction property - note also set > "unchecked_tombstone_compaction" to true. > > compaction = {'class': > 'org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy', > 'compaction_window_size': '7', 'compaction_window_unit': 'DAYS', > 'max_threshold': '4', 'min_threshold': '4', 'unchecked_tombstone_compaction': > 'true'} > > We insert data with timestamp and TTL programmatically. > > Here's one set of sstable that I expect to be removed: > > $ ls -lt *Data.db | tail -5 > -rw-r--r--. 1 vchadoop vchadoop 31245097312 Sep 20 17:16 mc-1308-big-Data.db > -rw-r--r--. 1 vchadoop vchadoop 31524316252 Sep 19 14:27 mc-1187-big-Data.db > -rw-r--r--. 1 vchadoop vchadoop 21405216502 Sep 18 14:14 mc-1070-big-Data.db > -rw-r--r--. 1 vchadoop vchadoop 13609890747 Sep 13 20:53 mc-178-big-Data.db > > $ date +%s > 1516895877 > > $ date > Thu Jan 25 15:58:00 UTC 2018 > > $ sstablemetadata $PWD/mc-130-big-Data.db | head -20 > SSTable: > /ae/disk1/data/ae/raw_logs_by_user-f58b9960980311e79ac26928246f09c1/mc-130-big > Partitioner: org.apache.cassandra.dht.Murmur3Partitioner > Bloom Filter FP chance: 0.01 > Minimum timestamp: 14966028 > Maximum timestamp: 14980788 > SSTable min local deletion time: 1507924954 > SSTable max local deletion time: 1509400832 > Compressor: org.apache.cassandra.io.compress.LZ4Compressor > Compression ratio: 0.17430158132352797 > TTL min: 2630598 > TTL max: 4086188 > First token: -9177441867697829836 (key=823134638755651936) > Last token: 9155171035305804798 (key=395118640769012487) > minClustringValues: [-1, da, 3, 1498082382078, -9223371818124305448, > -9223371652504795402, -1] > maxClustringValues: [61818, tpt, 325, 149660280, -4611686088173246790, > 9223372014135560885, 1] > Estimated droppable tombstones: 1.1983492967652476 > SSTable Level: 0 > Repaired at: 0 > Replay positions covered: {CommitLogPosition(segmentId=1505171071629, > position=7157684)=CommitLogPosition(segmentId=1505171075152, > position=6263269)} > totalColumnsSet: 111047277 >
Re: TWCS not deleting expired sstables
Hi Brian, Thanks for your response. Yes, I did look at that post. In fact after reading that post, I set the "unchecked_tombstone_compaction" to true. For the sstable in the example below (and its neighbors), all the data for those time windows have been compacted into a single sstable, so there is no dependency or delay caused from other sstables. Thanks, Jayesh From: Date: Sunday, January 28, 2018 at 1:02 PM To: Subject: Re: TWCS not deleting expired sstables I would start here: http://thelastpickle.com/blog/2016/12/08/TWCS-part1.html Specifically the “Hints and repairs” and “Timestamp overlap” sections might be of use. -B On Jan 25, 2018, at 11:05 AM, Thakrar, Jayesh mailto:jthak...@conversantmedia.com>> wrote: Wondering if I can get some pointers to what's happening here and why sstables that I think should be expired are not being dropped. Here's the table's compaction property - note also set "unchecked_tombstone_compaction" to true. compaction = {'class': 'org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy', 'compaction_window_size': '7', 'compaction_window_unit': 'DAYS', 'max_threshold': '4', 'min_threshold': '4', 'unchecked_tombstone_compaction': 'true'} We insert data with timestamp and TTL programmatically. Here's one set of sstable that I expect to be removed: $ ls -lt *Data.db | tail -5 -rw-r--r--. 1 vchadoop vchadoop 31245097312 Sep 20 17:16 mc-1308-big-Data.db -rw-r--r--. 1 vchadoop vchadoop 31524316252 Sep 19 14:27 mc-1187-big-Data.db -rw-r--r--. 1 vchadoop vchadoop 21405216502 Sep 18 14:14 mc-1070-big-Data.db -rw-r--r--. 1 vchadoop vchadoop 13609890747 Sep 13 20:53 mc-178-big-Data.db $ date +%s 1516895877 $ date Thu Jan 25 15:58:00 UTC 2018 $ sstablemetadata $PWD/mc-130-big-Data.db | head -20 SSTable: /ae/disk1/data/ae/raw_logs_by_user-f58b9960980311e79ac26928246f09c1/mc-130-big Partitioner: org.apache.cassandra.dht.Murmur3Partitioner Bloom Filter FP chance: 0.01 Minimum timestamp: 14966028 Maximum timestamp: 14980788 SSTable min local deletion time: 1507924954 SSTable max local deletion time: 1509400832 Compressor: org.apache.cassandra.io.compress.LZ4Compressor Compression ratio: 0.17430158132352797 TTL min: 2630598 TTL max: 4086188 First token: -9177441867697829836 (key=823134638755651936) Last token: 9155171035305804798 (key=395118640769012487) minClustringValues: [-1, da, 3, 1498082382078, -9223371818124305448, -9223371652504795402, -1] maxClustringValues: [61818, tpt, 325, 149660280, -4611686088173246790, 9223372014135560885, 1] Estimated droppable tombstones: 1.1983492967652476 SSTable Level: 0 Repaired at: 0 Replay positions covered: {CommitLogPosition(segmentId=1505171071629, position=7157684)=CommitLogPosition(segmentId=1505171075152, position=6263269)} totalColumnsSet: 111047277
Re: TWCS not deleting expired sstables
Likely a read repair caused old data to be brought into a newer SSTable. Try running sstableexpiredblockers to find out if there's a newer SSTable blocking that one from being dropped.
RE: TWCS not deleting expired sstables
Wow! It’s in the DataStax documentation: https://docs.datastax.com/en/dse/5.1/dse-admin/datastax_enterprise/tools/toolsSStables/toolsSStabExpiredBlockers.html Other nice tools there as well: https://docs.datastax.com/en/dse/5.1/dse-admin/datastax_enterprise/tools/toolsSStables/toolsSSTableUtilitiesTOC.html Kenneth Brotman From: kurt greaves [mailto:k...@instaclustr.com] Sent: Monday, January 29, 2018 8:20 PM To: User Subject: Re: TWCS not deleting expired sstables Likely a read repair caused old data to be brought into a newer SSTable. Try running sstableexpiredblockers to find out if there's a newer SSTable blocking that one from being dropped.
Re: TWCS not deleting expired sstables
Thanks Kurt and Kenneth. Now only if they would work as expected. node111.ord.ae.tsg.cnvr.net:/ae/disk1/data/ae/raw_logs_by_user-f58b9960980311e79ac26928246f09c1>ls -lt | tail -rw-r--r--. 1 vchadoop vchadoop286889260 Sep 18 14:14 mc-1070-big-Index.db -rw-r--r--. 1 vchadoop vchadoop12236 Sep 13 20:53 mc-178-big-Statistics.db -rw-r--r--. 1 vchadoop vchadoop 92 Sep 13 20:53 mc-178-big-TOC.txt -rw-r--r--. 1 vchadoop vchadoop 9371211 Sep 13 20:53 mc-178-big-CompressionInfo.db -rw-r--r--. 1 vchadoop vchadoop 10 Sep 13 20:53 mc-178-big-Digest.crc32 -rw-r--r--. 1 vchadoop vchadoop 13609890747 Sep 13 20:53 mc-178-big-Data.db -rw-r--r--. 1 vchadoop vchadoop 1394968 Sep 13 20:53 mc-178-big-Summary.db -rw-r--r--. 1 vchadoop vchadoop 11172592 Sep 13 20:53 mc-178-big-Filter.db -rw-r--r--. 1 vchadoop vchadoop190508739 Sep 13 20:53 mc-178-big-Index.db drwxr-xr-x. 2 vchadoop vchadoop 10 Sep 12 21:47 backups node111.ord.ae.tsg.cnvr.net:/ae/disk1/data/ae/raw_logs_by_user-f58b9960980311e79ac26928246f09c1>sstableexpiredblockers ae raw_logs_by_user Exception in thread "main" java.lang.IllegalArgumentException: Unknown keyspace/table ae.raw_logs_by_user at org.apache.cassandra.tools.SSTableExpiredBlockers.main(SSTableExpiredBlockers.java:66) node111.ord.ae.tsg.cnvr.net:/ae/disk1/data/ae/raw_logs_by_user-f58b9960980311e79ac26928246f09c1>sstableexpiredblockers system peers No sstables for system.peers node111.ord.ae.tsg.cnvr.net:/ae/disk1/data/ae/raw_logs_by_user-f58b9960980311e79ac26928246f09c1>ls -l ../../system/peers-37f71aca7dc2383ba70672528af04d4f/ total 308 drwxr-xr-x. 2 vchadoop vchadoop 10 Sep 11 22:59 backups -rw-rw-r--. 1 vchadoop vchadoop 83 Jan 25 02:11 mc-137-big-CompressionInfo.db -rw-rw-r--. 1 vchadoop vchadoop 180369 Jan 25 02:11 mc-137-big-Data.db -rw-rw-r--. 1 vchadoop vchadoop 10 Jan 25 02:11 mc-137-big-Digest.crc32 -rw-rw-r--. 1 vchadoop vchadoop 64 Jan 25 02:11 mc-137-big-Filter.db -rw-rw-r--. 1 vchadoop vchadoop386 Jan 25 02:11 mc-137-big-Index.db -rw-rw-r--. 1 vchadoop vchadoop 5171 Jan 25 02:11 mc-137-big-Statistics.db -rw-rw-r--. 1 vchadoop vchadoop 56 Jan 25 02:11 mc-137-big-Summary.db -rw-rw-r--. 1 vchadoop vchadoop 92 Jan 25 02:11 mc-137-big-TOC.txt -rw-rw-r--. 1 vchadoop vchadoop 43 Jan 29 21:11 mc-138-big-CompressionInfo.db -rw-rw-r--. 1 vchadoop vchadoop 9723 Jan 29 21:11 mc-138-big-Data.db -rw-rw-r--. 1 vchadoop vchadoop 10 Jan 29 21:11 mc-138-big-Digest.crc32 -rw-rw-r--. 1 vchadoop vchadoop 16 Jan 29 21:11 mc-138-big-Filter.db -rw-rw-r--. 1 vchadoop vchadoop 17 Jan 29 21:11 mc-138-big-Index.db -rw-rw-r--. 1 vchadoop vchadoop 5015 Jan 29 21:11 mc-138-big-Statistics.db -rw-rw-r--. 1 vchadoop vchadoop 56 Jan 29 21:11 mc-138-big-Summary.db -rw-rw-r--. 1 vchadoop vchadoop 92 Jan 29 21:11 mc-138-big-TOC.txt -rw-rw-r--. 1 vchadoop vchadoop 43 Jan 29 21:53 mc-139-big-CompressionInfo.db -rw-rw-r--. 1 vchadoop vchadoop 18908 Jan 29 21:53 mc-139-big-Data.db -rw-rw-r--. 1 vchadoop vchadoop 10 Jan 29 21:53 mc-139-big-Digest.crc32 -rw-rw-r--. 1 vchadoop vchadoop 16 Jan 29 21:53 mc-139-big-Filter.db -rw-rw-r--. 1 vchadoop vchadoop 36 Jan 29 21:53 mc-139-big-Index.db -rw-rw-r--. 1 vchadoop vchadoop 5055 Jan 29 21:53 mc-139-big-Statistics.db -rw-rw-r--. 1 vchadoop vchadoop 56 Jan 29 21:53 mc-139-big-Summary.db -rw-rw-r--. 1 vchadoop vchadoop 92 Jan 29 21:53 mc-139-big-TOC.txt From: Kenneth Brotman Date: Tuesday, January 30, 2018 at 7:37 AM To: Subject: RE: TWCS not deleting expired sstables Wow! It’s in the DataStax documentation: https://docs.datastax.com/en/dse/5.1/dse-admin/datastax_enterprise/tools/toolsSStables/toolsSStabExpiredBlockers.html Other nice tools there as well: https://docs.datastax.com/en/dse/5.1/dse-admin/datastax_enterprise/tools/toolsSStables/toolsSSTableUtilitiesTOC.html Kenneth Brotman From: kurt greaves [mailto:k...@instaclustr.com] Sent: Monday, January 29, 2018 8:20 PM To: User Subject: Re: TWCS not deleting expired sstables Likely a read repair caused old data to be brought into a newer SSTable. Try running sstableexpiredblockers to find out if there's a newer SSTable blocking that one from being dropped.
Re: TWCS not deleting expired sstables
Well, that shouldn't happen. Seems like it's possibly not looking in the correct location for data directories. Try setting CASSANDRA_INCLUDE= prior to running the script? e.g: CASSANDRA_INCLUDE=/cassandra.in.sh sstableexpiredblockers ae raw_logs_by_user On 30 January 2018 at 15:34, Thakrar, Jayesh wrote: > Thanks Kurt and Kenneth. > > > > Now only if they would work as expected. > > > > *node111.ord.ae.tsg.cnvr.net:/ae/disk1/data/ae/raw_logs_by_user-f58b9960980311e79ac26928246f09c1>ls > -lt | tail * > > -rw-r--r--. 1 vchadoop vchadoop286889260 Sep 18 14:14 > mc-1070-big-Index.db > > -rw-r--r--. 1 vchadoop vchadoop12236 Sep 13 20:53 > mc-178-big-Statistics.db > > -rw-r--r--. 1 vchadoop vchadoop 92 Sep 13 20:53 > mc-178-big-TOC.txt > > -rw-r--r--. 1 vchadoop vchadoop 9371211 Sep 13 20:53 > mc-178-big-CompressionInfo.db > > -rw-r--r--. 1 vchadoop vchadoop 10 Sep 13 20:53 > mc-178-big-Digest.crc32 > > -rw-r--r--. 1 vchadoop vchadoop 13609890747 <(360)%20989-0747> Sep 13 > 20:53 mc-178-big-Data.db > > -rw-r--r--. 1 vchadoop vchadoop 1394968 Sep 13 20:53 > mc-178-big-Summary.db > > -rw-r--r--. 1 vchadoop vchadoop 11172592 Sep 13 20:53 > mc-178-big-Filter.db > > -rw-r--r--. 1 vchadoop vchadoop190508739 Sep 13 20:53 > mc-178-big-Index.db > > drwxr-xr-x. 2 vchadoop vchadoop 10 Sep 12 21:47 backups > > > > *node111.ord.ae.tsg.cnvr.net:/ae/disk1/data/ae/raw_logs_by_user-f58b9960980311e79ac26928246f09c1>sstableexpiredblockers > ae raw_logs_by_user* > > Exception in thread "main" java.lang.IllegalArgumentException: Unknown > keyspace/table ae.raw_logs_by_user > > at org.apache.cassandra.tools.SSTableExpiredBlockers.main( > SSTableExpiredBlockers.java:66) > > > > *node111.ord.ae.tsg.cnvr.net:/ae/disk1/data/ae/raw_logs_by_user-f58b9960980311e79ac26928246f09c1>sstableexpiredblockers > system peers* > > No sstables for system.peers > > > > *node111.ord.ae.tsg.cnvr.net:/ae/disk1/data/ae/raw_logs_by_user-f58b9960980311e79ac26928246f09c1>ls > -l ../../system/peers-37f71aca7dc2383ba70672528af04d4f/* > > total 308 > > drwxr-xr-x. 2 vchadoop vchadoop 10 Sep 11 22:59 backups > > -rw-rw-r--. 1 vchadoop vchadoop 83 Jan 25 02:11 > mc-137-big-CompressionInfo.db > > -rw-rw-r--. 1 vchadoop vchadoop 180369 Jan 25 02:11 mc-137-big-Data.db > > -rw-rw-r--. 1 vchadoop vchadoop 10 Jan 25 02:11 mc-137-big-Digest.crc32 > > -rw-rw-r--. 1 vchadoop vchadoop 64 Jan 25 02:11 mc-137-big-Filter.db > > -rw-rw-r--. 1 vchadoop vchadoop386 Jan 25 02:11 mc-137-big-Index.db > > -rw-rw-r--. 1 vchadoop vchadoop 5171 Jan 25 02:11 > mc-137-big-Statistics.db > > -rw-rw-r--. 1 vchadoop vchadoop 56 Jan 25 02:11 mc-137-big-Summary.db > > -rw-rw-r--. 1 vchadoop vchadoop 92 Jan 25 02:11 mc-137-big-TOC.txt > > -rw-rw-r--. 1 vchadoop vchadoop 43 Jan 29 21:11 > mc-138-big-CompressionInfo.db > > -rw-rw-r--. 1 vchadoop vchadoop 9723 Jan 29 21:11 mc-138-big-Data.db > > -rw-rw-r--. 1 vchadoop vchadoop 10 Jan 29 21:11 mc-138-big-Digest.crc32 > > -rw-rw-r--. 1 vchadoop vchadoop 16 Jan 29 21:11 mc-138-big-Filter.db > > -rw-rw-r--. 1 vchadoop vchadoop 17 Jan 29 21:11 mc-138-big-Index.db > > -rw-rw-r--. 1 vchadoop vchadoop 5015 Jan 29 21:11 > mc-138-big-Statistics.db > > -rw-rw-r--. 1 vchadoop vchadoop 56 Jan 29 21:11 mc-138-big-Summary.db > > -rw-rw-r--. 1 vchadoop vchadoop 92 Jan 29 21:11 mc-138-big-TOC.txt > > -rw-rw-r--. 1 vchadoop vchadoop 43 Jan 29 21:53 > mc-139-big-CompressionInfo.db > > -rw-rw-r--. 1 vchadoop vchadoop 18908 Jan 29 21:53 mc-139-big-Data.db > > -rw-rw-r--. 1 vchadoop vchadoop 10 Jan 29 21:53 mc-139-big-Digest.crc32 > > -rw-rw-r--. 1 vchadoop vchadoop 16 Jan 29 21:53 mc-139-big-Filter.db > > -rw-rw-r--. 1 vchadoop vchadoop 36 Jan 29 21:53 mc-139-big-Index.db > > -rw-rw-r--. 1 vchadoop vchadoop 5055 Jan 29 21:53 > mc-139-big-Statistics.db > > -rw-rw-r--. 1 vchadoop vchadoop 56 Jan 29 21:53 mc-139-big-Summary.db > > -rw-rw-r--. 1 vchadoop vchadoop 92 Jan 29 21:53 mc-139-big-TOC.txt > > > > > > > > *From: *Kenneth Brotman > *Date: *Tuesday, January 30, 2018 at 7:37 AM > *To: * > *Subject: *RE: TWCS not deleting expired sstables > > > > Wow! It’s in the DataStax documentation: https://docs.datastax.com/en/ > dse/5.1/dse-admin/datastax_enterprise/tools/toolsSStables/ > toolsSStabExpiredBlockers.html > > > > Other nice tools there as well: https://docs.datastax.com/en/ > dse/5.1/dse-admin/datastax_enterprise/tools/toolsSStables/ > toolsSSTableUtilitiesTOC.html > > > > Kenneth Brotman > > > > *From:* kurt greaves [mailto:k...@instaclustr.com] > *Sent:* Monday, January 29, 2018 8:20 PM > *To:* User > *Subject:* Re: TWCS not deleting expired sstables > > > > Likely a read repair caused old data to be brought into a newer SSTable. > Try running sstableexpiredblockers to find out if there's a newer SSTable > blocking that one from being dropped. >
Re: TWCS not deleting expired sstables
Thank you so much Kurt - that helped!! So here's the thing, I was logged into the server as my own id and had Cassandra binaries under my home directory with the default configuration. Cassandra was running on the server with a service account and with its own user-id and configuration. I assumed that like nodetool, this command would also use the Cassandra service JMX or server port. However your suggestion made me realize that that's not the case. So I copied the service account's Cassandra/conf to my local install and things worked. I also looked up the source code and it did not indicate any "connection" via service or jmx port :) So now need to see why the data is not getting purged. Thanks again!! From: kurt greaves Date: Wednesday, January 31, 2018 at 11:27 PM To: User Subject: Re: TWCS not deleting expired sstables Well, that shouldn't happen. Seems like it's possibly not looking in the correct location for data directories. Try setting CASSANDRA_INCLUDE=http://cassandra.in.sh>> prior to running the script? e.g: CASSANDRA_INCLUDE=/cassandra.in.sh<http://cassandra.in.sh> sstableexpiredblockers ae raw_logs_by_user On 30 January 2018 at 15:34, Thakrar, Jayesh mailto:jthak...@conversantmedia.com>> wrote: Thanks Kurt and Kenneth. Now only if they would work as expected. node111.ord.ae.tsg.cnvr.net:/ae/disk1/data/ae/raw_logs_by_user-f58b9960980311e79ac26928246f09c1>ls -lt | tail -rw-r--r--. 1 vchadoop vchadoop286889260 Sep 18 14:14 mc-1070-big-Index.db -rw-r--r--. 1 vchadoop vchadoop12236 Sep 13 20:53 mc-178-big-Statistics.db -rw-r--r--. 1 vchadoop vchadoop 92 Sep 13 20:53 mc-178-big-TOC.txt -rw-r--r--. 1 vchadoop vchadoop 9371211 Sep 13 20:53 mc-178-big-CompressionInfo.db -rw-r--r--. 1 vchadoop vchadoop 10 Sep 13 20:53 mc-178-big-Digest.crc32 -rw-r--r--. 1 vchadoop vchadoop 13609890747 Sep 13 20:53 mc-178-big-Data.db -rw-r--r--. 1 vchadoop vchadoop 1394968 Sep 13 20:53 mc-178-big-Summary.db -rw-r--r--. 1 vchadoop vchadoop 11172592 Sep 13 20:53 mc-178-big-Filter.db -rw-r--r--. 1 vchadoop vchadoop190508739 Sep 13 20:53 mc-178-big-Index.db drwxr-xr-x. 2 vchadoop vchadoop 10 Sep 12 21:47 backups node111.ord.ae.tsg.cnvr.net:/ae/disk1/data/ae/raw_logs_by_user-f58b9960980311e79ac26928246f09c1>sstableexpiredblockers ae raw_logs_by_user Exception in thread "main" java.lang.IllegalArgumentException: Unknown keyspace/table ae.raw_logs_by_user at org.apache.cassandra.tools.SSTableExpiredBlockers.main(SSTableExpiredBlockers.java:66) node111.ord.ae.tsg.cnvr.net:/ae/disk1/data/ae/raw_logs_by_user-f58b9960980311e79ac26928246f09c1>sstableexpiredblockers system peers No sstables for system.peers node111.ord.ae.tsg.cnvr.net:/ae/disk1/data/ae/raw_logs_by_user-f58b9960980311e79ac26928246f09c1>ls -l ../../system/peers-37f71aca7dc2383ba70672528af04d4f/ total 308 drwxr-xr-x. 2 vchadoop vchadoop 10 Sep 11 22:59 backups -rw-rw-r--. 1 vchadoop vchadoop 83 Jan 25 02:11 mc-137-big-CompressionInfo.db -rw-rw-r--. 1 vchadoop vchadoop 180369 Jan 25 02:11 mc-137-big-Data.db -rw-rw-r--. 1 vchadoop vchadoop 10 Jan 25 02:11 mc-137-big-Digest.crc32 -rw-rw-r--. 1 vchadoop vchadoop 64 Jan 25 02:11 mc-137-big-Filter.db -rw-rw-r--. 1 vchadoop vchadoop386 Jan 25 02:11 mc-137-big-Index.db -rw-rw-r--. 1 vchadoop vchadoop 5171 Jan 25 02:11 mc-137-big-Statistics.db -rw-rw-r--. 1 vchadoop vchadoop 56 Jan 25 02:11 mc-137-big-Summary.db -rw-rw-r--. 1 vchadoop vchadoop 92 Jan 25 02:11 mc-137-big-TOC.txt -rw-rw-r--. 1 vchadoop vchadoop 43 Jan 29 21:11 mc-138-big-CompressionInfo.db -rw-rw-r--. 1 vchadoop vchadoop 9723 Jan 29 21:11 mc-138-big-Data.db -rw-rw-r--. 1 vchadoop vchadoop 10 Jan 29 21:11 mc-138-big-Digest.crc32 -rw-rw-r--. 1 vchadoop vchadoop 16 Jan 29 21:11 mc-138-big-Filter.db -rw-rw-r--. 1 vchadoop vchadoop 17 Jan 29 21:11 mc-138-big-Index.db -rw-rw-r--. 1 vchadoop vchadoop 5015 Jan 29 21:11 mc-138-big-Statistics.db -rw-rw-r--. 1 vchadoop vchadoop 56 Jan 29 21:11 mc-138-big-Summary.db -rw-rw-r--. 1 vchadoop vchadoop 92 Jan 29 21:11 mc-138-big-TOC.txt -rw-rw-r--. 1 vchadoop vchadoop 43 Jan 29 21:53 mc-139-big-CompressionInfo.db -rw-rw-r--. 1 vchadoop vchadoop 18908 Jan 29 21:53 mc-139-big-Data.db -rw-rw-r--. 1 vchadoop vchadoop 10 Jan 29 21:53 mc-139-big-Digest.crc32 -rw-rw-r--. 1 vchadoop vchadoop 16 Jan 29 21:53 mc-139-big-Filter.db -rw-rw-r--. 1 vchadoop vchadoop 36 Jan 29 21:53 mc-139-big-Index.db -rw-rw-r--. 1 vchadoop vchadoop 5055 Jan 29 21:53 mc-139-big-Statistics.db -rw-rw-r--. 1 vchadoop vchadoop 56 Jan 29 21:53 mc-139-big-Summary.db -rw-rw-r--. 1 vchadoop vchadoop 92 Jan 29 21:53 mc-139-big-TOC.txt From: Kenneth Brotman Date: Tuesday, January 30, 2018 at 7:37 AM To: mailto:user@cassandra.apache.org>> Subject: RE: TWCS not deleting expired sstables Wo