[
https://issues.apache.org/jira/browse/CASSANDRA-11412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15237034#comment-15237034
]
Marcus Eriksson edited comment on CASSANDRA-11412 at 4/12/16 11:54 AM:
-----------------------------------------------------------------------
My thinking is that for non-LCS workloads "all" sstables cover the whole range
- it should be quite rare that we have any benefit from doing something like
you suggest - reason being that with other compaction strategies we never split
based on range
Now, this is not entirely true of course, say temporarily after a bootstrap or
a repair we could have many non-overlapping sstables, but the gain should be
small and during a short time span because:
* we never compact more than 32 sstables together so during compaction we open
at most 32 scanners
* repair opens all sstables, but in the incremental world we only open
unrepaired sstables, while the many streamed in sstables after a repair are
already repaired.
Does that make sense? I think we should do the improvement, but in another
ticket as it is not as urgent as this
was (Author: krummas):
My thinking is that for non-LCS workloads "all" sstables cover the whole range
- it should be quite rare that we have any benefit from doing something like
you suggest - reason being that with other compaction strategies we never split
based on range - "all" sstables cover the whole range.
Now, this is not entirely true of course, say temporarily after a bootstrap or
a repair we could have many non-overlapping sstables, but the gain should be
small and during a short time span because:
* we never compact more than 32 sstables together so during compaction we open
at most 32 scanners
* repair opens all sstables, but in the incremental world we only open
unrepaired sstables, while the many streamed in sstables after a repair are
already repaired.
Does that make sense? I think we should do the improvement, but in another
ticket as it is not as urgent as this
> Many sstablescanners opened during repair
> -----------------------------------------
>
> Key: CASSANDRA-11412
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11412
> Project: Cassandra
> Issue Type: Improvement
> Reporter: Marcus Eriksson
> Assignee: Marcus Eriksson
> Fix For: 3.0.x, 3.x
>
>
> Since CASSANDRA-5220 we open [one sstablescanner per range per
> sstable|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/db/compaction/CompactionStrategyManager.java#L374].
> If compaction gets behind and you are running vnodes with 256 tokens and
> RF3, this could become a problem (ie, {{768 * number of sstables}} scanners)
> We could probably refactor this similar to the way we handle scanners with
> LCS - only open the scanner once we need it
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
