[
https://issues.apache.org/jira/browse/CASSANDRA-3411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13138060#comment-13138060
]
Rick Branson commented on CASSANDRA-3411:
-----------------------------------------
So as soon as the number of spare segments drops to 0, we'll pre-allocate a new
one. It ties in with the last point you made about using a separate thread,
which is a no brainer. This sounds like a good strategy, and should smooth out
some of the latency spikes.
I like the idea of throwing away the extra config variable, but I'm not sure if
I'd like to save the high watermark to the system table. An anomaly could lead
this to getting permanently set too high and riding up against the total
commitlog size limit. We'd have to provide some user-accessible way to adjust
the high watermark at that point or add some "intelligence" that could bring
the high watermark back down.
I propose an initial 4 segments (or up to the total commitlog size limit,
whichever is lower) at startup, allocating more as needed, but no longer
discarding them. Combined with the eager background pre-allocation, the vast
majority of use cases should quickly and smoothly reach a steady state.
I'll also drag my way back to svn-land ;)
> Pre-allocated, Recycled Commitlog Segment Files
> -----------------------------------------------
>
> Key: CASSANDRA-3411
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3411
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Rick Branson
> Priority: Minor
> Attachments:
> 001-pre-allocated-recycled-commitlog-segment-files-CASSANDRA-3411.patch,
> 3411-cleaned.txt
>
>
> An approach for improving commitlog performance is to pre-allocate the full
> 128MB segment files and reuse them once all the mutations have been flushed.
> Pre-allocation allows writes to be performed without modifying the file size
> metadata, and should (in theory) allow the filesystem to allocate a
> contiguous block of space for the file. Recycling the segment files prevents
> the overhead of pre-allocation from impacting overall performance.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
