We're running C* 1.2.11 and have two CFs, one called OAuth2AccessToken and one OAuth2AccessTokensByUser. OAuth2AccessToken has the token as the row key, and the columns are some data about the OAuth token. There's a TTL set on it, usually 3600, but can be higher (up to 1 month). OAuth2AccessTokensByUser has the user as the row key, and then all of the user's token identifiers as column values. Each of the column values has a TTL that is set to the same as the access token it corresponds to.
The OAuth2AccessToken CF takes up around ~6 GB on disk, whereas the OAuth2AccessTokensByUser CF takes around ~110 GB. If I use sstablemetadata, I can see the droppable tombstones ratio is around 90% for the larger sstables. My question is - why aren't these tombstones getting compacted away? I'm guessing that it's because we use STCS and the large sstables that have built up over time are never considered for compaction. Would LCS be a better fit for the issue of trying to keep the tombstones in check? I've also tried forceUserDefinedCompaction via JMX on some of the largest sstables and it just creates a new sstable of the exact same size, which was pretty surprising. Why would this explicit request to compact an sstable not remove tombstones? Thanks! Kevin
