[
https://issues.apache.org/jira/browse/CASSANDRA-12464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16458637#comment-16458637
]
Iuri Sitinschi commented on CASSANDRA-12464:
--------------------------------------------
Link in the description is broken. It should be
https://github.com/google/leveldb/blob/master/doc/impl.md#compactions
> Investigate the potential improvement of parallelism on higher level
> compactions in LCS
> ---------------------------------------------------------------------------------------
>
> Key: CASSANDRA-12464
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12464
> Project: Cassandra
> Issue Type: Improvement
> Components: Compaction
> Reporter: Wei Deng
> Priority: Major
> Labels: lcs, lhf, performance
>
> According to LevelDB's design doc
> [here|https://github.com/google/leveldb/blob/master/doc/impl.html#L115-L116],
> "A compaction merges the contents of the picked files to produce a sequence
> of level-(L+1) files", it will "switch to producing a new level-(L+1) file
> after the current output file has reached the target file size" (in our case
> 160MB), it will also "switch to a new output file when the key range of the
> current output file has grown enough to overlap more than ten level-(L+2)
> files". This is to ensure "that a later compaction of a level-(L+1) file will
> not pick up too much data from level-(L+2)."
> Our current code in LeveledCompactionStrategy doesn't implement this last
> rule, but we might be able to quickly implement it and see how much a
> compaction throughput improvement it can deliver. Potentially we can create a
> scenario where a number of large L0 SSTables are present (e.g. 200GB after
> switching from STCS) and let it to create thousands of L1 SSTables overflow,
> and see how fast LCS can digest this much data from L1 and properly
> upper-level them to completion.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
