/* Sorry for the mess, accidental Ctrl-Enter */ Fellow C* users,
I've got a cluster of C* 3.5 serving a single keyspace with DTCS table and no deletes. We knew that data does not expire on time (even after gc_grace_period) - that's sth we wanted to investigate later and eventually let C* to keep it longer. The moment when C* decided to remove old data happened yesterday and on a one node we've reduced the data size by 40% (removing ~350 SSTables). On remaining nodes that was only ~5%. That's of course wrong and all nodes should get down by 40%. A closer look and we really found that ~40% is out-of-TTL and that maps to ~350 SSTables. I've triggered a user defined compaction on few of the SStables but ... nodetool compact exited with 0 a second after and file remain untouched (no entry in the debug.log about compaction). The files are mmap()ed in the process as well. Is there a known bug for this behaviour? Why one node removed *all* data at once (there's data that should expire 3 months ago) while others not? All nodes are equal in spec and configuration. -Jacek 2016-12-08 14:11 GMT+01:00 Jacek Luczak <difrost.ker...@gmail.com>: > Fellow C* users, > > I've got a cluster of C* 3.5 serving a single keyspace with DTCS table > and no deletes. We knew that data does not expire on time (even after > gc_grace_period) - that's sth we wanted to investigate later and > eventually let C* to keep it longer. The moment when C* decided to > remove old data happened yesterday and on a one node we've reduced the > data size by 40% (removing ~350 SSTables). On remaining nodes that was > only ~5%. That's of course wrong and all nodes should get down by 40%. > A closer look and we really found that ~40% is out-of-TTL and that > maps to ~350 SSTables. > > I've triggered a user defined compaction on few of the tables but ... > nodetool compact