[
https://issues.apache.org/jira/browse/CASSANDRA-1470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12970225#action_12970225
]
T Jake Luciani commented on CASSANDRA-1470:
-------------------------------------------
.bq see http://chbits.blogspot.com/2010/06/lucene-and-fadvisemadvise.html for
why posix_fadvise won't work [for writes]
This article is talking about NOREUSE flag being a no-op but we are using
DONTNEED which does work.
Since the true goal of this ticket is to minimize the performance impact of
compaction I'd like to try the following:
At BRAF level:
* use fadvise(DONTNEED) instead of direct-io for writes. This will fix the
buffering problem we now see affecting write speed.
* use fadvise(DONTNEED) for sstable reads to remove the need for directio
flag altogether.
* add a method long[] pagesInPageCache() which uses the posix mincore()
function to detect the offsets of pages for this file currently in page cache.
At Compaction level(a separate ticket):
* add getActiveKeys() which uses underlying pagesInPageCache() to get the
keys actually in the page cache
* use getActiveKeys() to detect which SSTables being compacted are in the os
cache and make sure the subsequent pages in the new compacted SSTable are kept
in the page cache for these keys. This will minimize the impact of compacting a
"hot" SSTable as well.
A simpler yet similar approach is described here:
http://insights.oetiker.ch/linux/fadvise/
> use direct io for compaction
> ----------------------------
>
> Key: CASSANDRA-1470
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1470
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Pavel Yaskevich
> Fix For: 0.7.1
>
> Attachments: 1470-v2.txt, 1470.txt, CASSANDRA-1470-for-0.6.patch,
> CASSANDRA-1470-v10-for-0.7.patch, CASSANDRA-1470-v11-for-0.7.patch,
> CASSANDRA-1470-v12-0.7.patch, CASSANDRA-1470-v2.patch,
> CASSANDRA-1470-v3-0.7-with-LastErrorException-support.patch,
> CASSANDRA-1470-v4-for-0.7.patch, CASSANDRA-1470-v5-for-0.7.patch,
> CASSANDRA-1470-v6-for-0.7.patch, CASSANDRA-1470-v7-for-0.7.patch,
> CASSANDRA-1470-v8-for-0.7.patch, CASSANDRA-1470-v9-for-0.7.patch,
> CASSANDRA-1470.patch,
> use.DirectIORandomAccessFile.for.commitlog.against.1022235.patch
>
>
> When compaction scans through a group of sstables, it forces the data in the
> os buffer cache being used for hot reads, which can have a dramatic negative
> effect on performance.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
