[ https://issues.apache.org/jira/browse/CASSANDRA-11159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15143939#comment-15143939 ]
Pavel Yaskevich commented on CASSANDRA-11159: --------------------------------------------- [~beobal] I ended up working on this today and it looks like we can't get away with simply switching index memtable after Tracker::replaceFlushed updated the view, because it would mean that some of the data from current memtable is going to be erased prematurely, ColumnIndex needs to keep a list of memtables pending flush much like View. I've added two additional notifications - switched and discarded and a SASI test to make sure that everything is properly switched/discarded to [CASSANDRA-11159|https://github.com/xedin/cassandra/tree/CASSANDRA-11159] branch. Please take a look and let me know what you think. > SASI indexes don't switch memtable on flush > ------------------------------------------- > > Key: CASSANDRA-11159 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11159 > Project: Cassandra > Issue Type: Bug > Components: Local Write-Read Paths > Reporter: Sam Tunnicliffe > Assignee: Sam Tunnicliffe > Priority: Critical > Fix For: 3.4 > > > SASI maintains its own in-memory structures for indexing the contents of a > base Memtable. On flush, these are simply discarded & replaced with an new > instance, whilst the on disk index is built as the base memtable is flushed > to SSTables. > SASIIndex implements INotificationHandler and this switching of the index > memtable is triggered by receipt of a MemtableRenewedNotification. In the > original SASI implementation, one of the necessary modifications to C* was to > emit this notification from DataTracker::switchMemtable, but this was > overlooked when porting to 3.0. The net result is that the index memtable is > never switched out, which eventually leads to OOME. > Simply applying the original modification isn't entirely appropriate though, > as it creates a window where it's possible for the index memtable to have > been switched, but the flushwriter is yet to finish writing the new index > sstables. During this window, index entries will be missing and query results > inaccurate. > I propose leaving Tracker::switchMemtable as is, so that > INotificationConsumers are only notified from there when truncating, but > adding similar notifications in Tracker::replaceFlushed, to fire after the > View is updated. > I'm leaning toward re-using MemtableRenewedNotification for this as > semantically I don't believe there's any meaningful difference between the > flush and truncation cases here. If anyone has a compelling argument for a > new notification type though to distinguish the two events, I'm open to hear > it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)