[
https://issues.apache.org/jira/browse/CASSANDRA-4781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yuki Morishita updated CASSANDRA-4781:
--------------------------------------
Attachment: 4781.txt
I found the cause of this recursive compaction.
We calculate how many columns in sstable are expected to be dropped, and if
that exceeds threshold we do single sstable compaction. Cassandra does not drop
tombstones when overlapping keys exist in other sstables, so when we have
overlap, we calculate what percentage of columns that do not belong to
overlapped keys can be dropped.
Here we use sstable.estimatedKeys to calculate, but since the value comes from
index interval and index summary, we have chance to calculate wrong value if
the number of keys in sstable is small, like in this case, schema_columns.
(schema_columns also has lot of overlapped keys among sstable since its key is
Keyspace name.)
So I propose to skip single sstable compaction if sstable contains small number
of keys.
Patch is attached for this.
> Sometimes Cassandra starts compacting system-shema_columns cf repeatedly
> until the node is killed
> -------------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-4781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4781
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 1.2.0 beta 1
> Environment: Ubuntu 12.04, single-node Cassandra cluster
> Reporter: Aleksey Yeschenko
> Assignee: Yuki Morishita
> Attachments: 4781.txt
>
>
> Cassandra starts flushing system-schema_columns cf in a seemingly infinite
> loop:
> INFO [CompactionExecutor:7] 2012-10-09 17:55:46,804 CompactionTask.java
> (line 239) Compacted to
> [/var/lib/cassandra/data/system/schema_columns/system-schema_columns-ia-32107-Data.db,].
> 3,827 to 3,827 (~100% of original) bytes for 3 keys at 0.202762MB/s. Time:
> 18ms.
> INFO [CompactionExecutor:7] 2012-10-09 17:55:46,804 CompactionTask.java
> (line 119) Compacting
> [SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-ia-32107-Data.db')]
> INFO [CompactionExecutor:7] 2012-10-09 17:55:46,824 CompactionTask.java
> (line 239) Compacted to
> [/var/lib/cassandra/data/system/schema_columns/system-schema_columns-ia-32108-Data.db,].
> 3,827 to 3,827 (~100% of original) bytes for 3 keys at 0.182486MB/s. Time:
> 20ms.
> INFO [CompactionExecutor:7] 2012-10-09 17:55:46,825 CompactionTask.java
> (line 119) Compacting
> [SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-ia-32108-Data.db')]
> INFO [CompactionExecutor:7] 2012-10-09 17:55:46,864 CompactionTask.java
> (line 239) Compacted to
> [/var/lib/cassandra/data/system/schema_columns/system-schema_columns-ia-32109-Data.db,].
> 3,827 to 3,827 (~100% of original) bytes for 3 keys at 0.096045MB/s. Time:
> 38ms.
> INFO [CompactionExecutor:7] 2012-10-09 17:55:46,864 CompactionTask.java
> (line 119) Compacting
> [SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-ia-32109-Data.db')]
> INFO [CompactionExecutor:7] 2012-10-09 17:55:46,894 CompactionTask.java
> (line 239) Compacted to
> [/var/lib/cassandra/data/system/schema_columns/system-schema_columns-ia-32110-Data.db,].
> 3,827 to 3,827 (~100% of original) bytes for 3 keys at 0.121657MB/s. Time:
> 30ms.
> INFO [CompactionExecutor:7] 2012-10-09 17:55:46,894 CompactionTask.java
> (line 119) Compacting
> [SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-ia-32110-Data.db')]
> INFO [CompactionExecutor:7] 2012-10-09 17:55:46,914 CompactionTask.java
> (line 239) Compacted to
> [/var/lib/cassandra/data/system/schema_columns/system-schema_columns-ia-32111-Data.db,].
> 3,827 to 3,827 (~100% of original) bytes for 3 keys at 0.202762MB/s. Time:
> 18ms.
> INFO [CompactionExecutor:7] 2012-10-09 17:55:46,914 CompactionTask.java
> (line 119) Compacting
> [SSTableReader(path='/var/lib/cassandra/data/system/schema_columns/system-schema_columns-ia-32111-Data.db')]
> .........
> Don't know what's causing it. Don't know a way to predictably trigger this
> behaviour. It just happens sometimes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
