Having a real issue where at the completion of large compactions, it will evict hot sstables from the kernel page cache causing huge read latency while it is backfilled.
https://dl.dropboxusercontent.com/s/149h7ssru0dapkg/Screen%20Shot%202013-07-12%20at%201.46.19%20PM.png Blue line - page cache Green line - disk read latency (ms) Red line - CF read latency (ms) The beginning of both high latency plateaus correspond with the completion of a compaction. Seems like applying/enabling this will help? https://issues.apache.org/jira/browse/CASSANDRA-4937 - C* 1.2.6 - 3 Nodes - 24G RAM (8G heap) - (2) 3TB 7.2k disks using JBOD feature of C*
