[ https://issues.apache.org/jira/browse/CASSANDRA-8099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14624267#comment-14624267 ]
Sylvain Lebresne commented on CASSANDRA-8099: --------------------------------------------- It is certainly not the intention to undo CASSANDRA-4858, but I could use some more details on why you think this has been undone. A priori the code from that ticket, which basically merge ranges that can be merged together, is still there in {{StorageProxy.RangeMerger}}. So did you experimentally noticed that this wasn't working as intended anymore? And if so, can you share your experiment? > Refactor and modernize the storage engine > ----------------------------------------- > > Key: CASSANDRA-8099 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8099 > Project: Cassandra > Issue Type: Improvement > Reporter: Sylvain Lebresne > Assignee: Sylvain Lebresne > Fix For: 3.0 beta 1 > > Attachments: 8099-nit > > > The current storage engine (which for this ticket I'll loosely define as "the > code implementing the read/write path") is suffering from old age. One of the > main problem is that the only structure it deals with is the cell, which > completely ignores the more high level CQL structure that groups cell into > (CQL) rows. > This leads to many inefficiencies, like the fact that during a reads we have > to group cells multiple times (to count on replica, then to count on the > coordinator, then to produce the CQL resultset) because we forget about the > grouping right away each time (so lots of useless cell names comparisons in > particular). But outside inefficiencies, having to manually recreate the CQL > structure every time we need it for something is hindering new features and > makes the code more complex that it should be. > Said storage engine also has tons of technical debt. To pick an example, the > fact that during range queries we update {{SliceQueryFilter.count}} is pretty > hacky and error prone. Or the overly complex ways {{AbstractQueryPager}} has > to go into to simply "remove the last query result". > So I want to bite the bullet and modernize this storage engine. I propose to > do 2 main things: > # Make the storage engine more aware of the CQL structure. In practice, > instead of having partitions be a simple iterable map of cells, it should be > an iterable list of row (each being itself composed of per-column cells, > though obviously not exactly the same kind of cell we have today). > # Make the engine more iterative. What I mean here is that in the read path, > we end up reading all cells in memory (we put them in a ColumnFamily object), > but there is really no reason to. If instead we were working with iterators > all the way through, we could get to a point where we're basically > transferring data from disk to the network, and we should be able to reduce > GC substantially. > Please note that such refactor should provide some performance improvements > right off the bat but it's not it's primary goal either. It's primary goal is > to simplify the storage engine and adds abstraction that are better suited to > further optimizations. -- This message was sent by Atlassian JIRA (v6.3.4#6332)