[
https://issues.apache.org/jira/browse/CASSANDRA-8641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14281457#comment-14281457
]
Brandon Williams commented on CASSANDRA-8641:
---------------------------------------------
Yes, and then the number of sstables makes sense, if a large portion of the
vnodes is damaged.
> Repair causes a large number of tiny SSTables
> ---------------------------------------------
>
> Key: CASSANDRA-8641
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8641
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Ubuntu 14.04
> Reporter: Flavien Charlon
> Fix For: 2.1.3
>
>
> I have a 3 nodes cluster with RF = 3, quad core and 32 GB or RAM. I am
> running 2.1.2 with all the default settings. I'm seeing some strange
> behaviors during incremental repair (under write load).
> Taking the example of one particular column family, before running an
> incremental repair, I have about 13 SSTables. After finishing the incremental
> repair, I have over 114000 SSTables.
> {noformat}
> Table: customers
> SSTable count: 114688
> Space used (live): 97203707290
> Space used (total): 99175455072
> Space used by snapshots (total): 0
> SSTable Compression Ratio: 0.28281112416526505
> Memtable cell count: 0
> Memtable data size: 0
> Memtable switch count: 1069
> Local read count: 0
> Local read latency: NaN ms
> Local write count: 11548705
> Local write latency: 0.030 ms
> Pending flushes: 0
> Bloom filter false positives: 0
> Bloom filter false ratio: 0.00000
> Bloom filter space used: 144145152
> Compacted partition minimum bytes: 311
> Compacted partition maximum bytes: 1996099046
> Compacted partition mean bytes: 3419
> Average live cells per slice (last five minutes): 0.0
> Maximum live cells per slice (last five minutes): 0.0
> Average tombstones per slice (last five minutes): 0.0
> Maximum tombstones per slice (last five minutes): 0.0
> {noformat}
> Looking at the logs during the repair, it seems Cassandra is struggling to
> compact minuscule memtables (often just a few kilobytes):
> {noformat}
> INFO [CompactionExecutor:337] 2015-01-17 01:44:27,011
> CompactionTask.java:251 - Compacted 32 sstables to
> [/mnt/data/cassandra/data/business/customers-d9d42d209ccc11e48ca54553c90a9d45/business-customers-ka-228341,].
> 8,332 bytes to 6,547 (~78% of original) in 80,476ms = 0.000078MB/s. 32
> total partitions merged to 32. Partition merge counts were {1:32, }
> INFO [CompactionExecutor:337] 2015-01-17 01:45:35,519
> CompactionTask.java:251 - Compacted 32 sstables to
> [/mnt/data/cassandra/data/business/customers-d9d42d209ccc11e48ca54553c90a9d45/business-customers-ka-229348,].
> 8,384 bytes to 6,563 (~78% of original) in 6,880ms = 0.000910MB/s. 32
> total partitions merged to 32. Partition merge counts were {1:32, }
> INFO [CompactionExecutor:339] 2015-01-17 01:47:46,475
> CompactionTask.java:251 - Compacted 32 sstables to
> [/mnt/data/cassandra/data/business/customers-d9d42d209ccc11e48ca54553c90a9d45/business-customers-ka-229351,].
> 8,423 bytes to 6,401 (~75% of original) in 10,416ms = 0.000586MB/s. 32
> total partitions merged to 32. Partition merge counts were {1:32, }
> {noformat}
>
> At the end of the repair, the cluster has become unusable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
