[
https://issues.apache.org/jira/browse/CASSANDRA-11159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15143939#comment-15143939
]
Pavel Yaskevich commented on CASSANDRA-11159:
---------------------------------------------
[~beobal] I ended up working on this today and it looks like we can't get away
with simply switching index memtable after Tracker::replaceFlushed updated the
view, because it would mean that some of the data from current memtable is
going to be erased prematurely, ColumnIndex needs to keep a list of memtables
pending flush much like View. I've added two additional notifications -
switched and discarded and a SASI test to make sure that everything is properly
switched/discarded to
[CASSANDRA-11159|https://github.com/xedin/cassandra/tree/CASSANDRA-11159]
branch. Please take a look and let me know what you think.
> SASI indexes don't switch memtable on flush
> -------------------------------------------
>
> Key: CASSANDRA-11159
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11159
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths
> Reporter: Sam Tunnicliffe
> Assignee: Sam Tunnicliffe
> Priority: Critical
> Fix For: 3.4
>
>
> SASI maintains its own in-memory structures for indexing the contents of a
> base Memtable. On flush, these are simply discarded & replaced with an new
> instance, whilst the on disk index is built as the base memtable is flushed
> to SSTables.
> SASIIndex implements INotificationHandler and this switching of the index
> memtable is triggered by receipt of a MemtableRenewedNotification. In the
> original SASI implementation, one of the necessary modifications to C* was to
> emit this notification from DataTracker::switchMemtable, but this was
> overlooked when porting to 3.0. The net result is that the index memtable is
> never switched out, which eventually leads to OOME.
> Simply applying the original modification isn't entirely appropriate though,
> as it creates a window where it's possible for the index memtable to have
> been switched, but the flushwriter is yet to finish writing the new index
> sstables. During this window, index entries will be missing and query results
> inaccurate.
> I propose leaving Tracker::switchMemtable as is, so that
> INotificationConsumers are only notified from there when truncating, but
> adding similar notifications in Tracker::replaceFlushed, to fire after the
> View is updated.
> I'm leaning toward re-using MemtableRenewedNotification for this as
> semantically I don't believe there's any meaningful difference between the
> flush and truncation cases here. If anyone has a compelling argument for a
> new notification type though to distinguish the two events, I'm open to hear
> it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
