Removing expired columns actually requires two compaction passes: one to turn the expired column into a tombstone; one to remove the tombstone after gc_grace_seconds. (See https://issues.apache.org/jira/browse/CASSANDRA-1537.)
Perhaps CASSANDRA-2786 was causing things to (erroneously) be cleaned up early enough that this helped you out in 0.8.2? On Wed, Mar 21, 2012 at 8:38 PM, Ross Black <ross.w.bl...@gmail.com> wrote: > Hi, > > We recently moved from 0.8.2 to 1.0.8 and the behaviour seems to have > changed so that tombstones are now not being deleted. > > Our application continually adds and removes columns from Cassandra. We > have set a short gc_grace time (3600) since our application would > automatically delete zombies if they appear. > Under 0.8.2, the tombstones remained at a relatively constant number. > Under 1.0.8, the tombstones have been continually increasing so that they > exceed the size of our real data (at this stage we have over 100G of > tombstones). > Even after running a full compact the new compacted SSTable contains a > massive number of tombstones, many that are several weeks old. > > Have I missed some new configuration option to allow deletion of tombstones? > > I also noticed that one of the changes between 0.8.2 and 1.0.8 was > https://issues.apache.org/jira/browse/CASSANDRA-2786 which changed code to > "avoid dropping tombstones when they might still be needed to shadow data in > another sstable". > Could this be having an impact since we continually add and remove columns > even while a major compact is executing? > > > Thanks, > Ross > -- Jonathan Ellis Project Chair, Apache Cassandra co-founder of DataStax, the source for professional Cassandra support http://www.datastax.com
