Hi Lahiru,
2.1.0 is also quite old (Sep 2014) - and just from my memory I
remembered that there was an issue whe had with cold_reads_to_omit:
http://grokbase.com/t/cassandra/user/1523sm4y0r/how-to-deal-with-too-many-sstables
https://www.mail-archive.com/search?l=user@cassandra.apache.org&q=subject:%22Re%3A+Compaction+failing+to+trigger%22&o=newest&f=1
That's just a random google hits but maybe that also helps.
I ended up with a few thousand sstables smaller than 1MB in size.
However I would suggest upgrading to a newer version of cassandra first
before diving too deep into this - maybe 2.1.16 or 2.2.8 - as chances
are really good your problems will be gone after that.
Regards.
Jan
