[ https://issues.apache.org/jira/browse/CASSANDRA-11412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15237034#comment-15237034 ]
Marcus Eriksson edited comment on CASSANDRA-11412 at 4/12/16 11:54 AM: ----------------------------------------------------------------------- My thinking is that for non-LCS workloads "all" sstables cover the whole range - it should be quite rare that we have any benefit from doing something like you suggest - reason being that with other compaction strategies we never split based on range Now, this is not entirely true of course, say temporarily after a bootstrap or a repair we could have many non-overlapping sstables, but the gain should be small and during a short time span because: * we never compact more than 32 sstables together so during compaction we open at most 32 scanners * repair opens all sstables, but in the incremental world we only open unrepaired sstables, while the many streamed in sstables after a repair are already repaired. Does that make sense? I think we should do the improvement, but in another ticket as it is not as urgent as this was (Author: krummas): My thinking is that for non-LCS workloads "all" sstables cover the whole range - it should be quite rare that we have any benefit from doing something like you suggest - reason being that with other compaction strategies we never split based on range - "all" sstables cover the whole range. Now, this is not entirely true of course, say temporarily after a bootstrap or a repair we could have many non-overlapping sstables, but the gain should be small and during a short time span because: * we never compact more than 32 sstables together so during compaction we open at most 32 scanners * repair opens all sstables, but in the incremental world we only open unrepaired sstables, while the many streamed in sstables after a repair are already repaired. Does that make sense? I think we should do the improvement, but in another ticket as it is not as urgent as this > Many sstablescanners opened during repair > ----------------------------------------- > > Key: CASSANDRA-11412 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11412 > Project: Cassandra > Issue Type: Improvement > Reporter: Marcus Eriksson > Assignee: Marcus Eriksson > Fix For: 3.0.x, 3.x > > > Since CASSANDRA-5220 we open [one sstablescanner per range per > sstable|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/db/compaction/CompactionStrategyManager.java#L374]. > If compaction gets behind and you are running vnodes with 256 tokens and > RF3, this could become a problem (ie, {{768 * number of sstables}} scanners) > We could probably refactor this similar to the way we handle scanners with > LCS - only open the scanner once we need it -- This message was sent by Atlassian JIRA (v6.3.4#6332)