That's basically the approach I want to take in https://issues.apache.org/jira/browse/CASSANDRA-2427.
On Thu, May 19, 2011 at 12:00 PM, Mike Malone <m...@simplegeo.com> wrote: > Just noticed this thread and figured I'd chime in since we've had similar > issues with the commit log growing too large on our clusters. Tuning down > the flush timeout wasn't really an acceptable solution for us since we > didn't want to be constantly flushing and generating extra SSTables for no > reason. So we wrote a small tool that we start in a static block in > CassandraServer that periodically checks the commit log size and flushes all > memtables if they're above some threshold. > I've attached that code. Any feedback / improvements are more than welcome. > > Mike > > On Thu, May 12, 2011 at 11:30 AM, Sanjeev Kulkarni <sanj...@locomatix.com> > wrote: >> >> Hey guys, >> I have a ec2 debian cluster consisting of several nodes running 0.7.5 on >> ephimeral disks. >> These are fresh installs and not upgrades. >> The commitlog is set to the smaller of the disks which is around 10G in >> size and the datadir is set to the bigger disk. >> The config file is basically the same as the one supplied by the default >> installation. >> Our applications write to the cluster. After about a day of writing we >> started noticing the commitlog disk filling up. Soon we went over the disk >> limit and writes started failing. At this point we stopped the cluster. >> Over the course of the day we inserted around 25G of data. Our columns >> values are pretty small. >> I understand that cassandra periodically cleans up the commitlog >> directories by generating sstables in datadir. Is there any way to speed up >> this movement from commitog to datadir? >> Thanks! > > -- Jonathan Ellis Project Chair, Apache Cassandra co-founder of DataStax, the source for professional Cassandra support http://www.datastax.com
