[
https://issues.apache.org/jira/browse/CASSANDRA-5417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13626622#comment-13626622
]
Sylvain Lebresne commented on CASSANDRA-5417:
---------------------------------------------
bq. Why do we need AbstractCompositeType now that we use AbstractCType?
One reason is that people may use a CompositeType as column value validator for
instance. Removing it would break those. And as of this patch, CompositeType is
still used for the row key even if it's composite. On that last part, I do
think we should replace it by CType internally at some point because that would
make sense, but I felt the current patch was big enough currently that this
could wait a followup ticket.
Another reason is that it's simpler for compatibility. The fact is, even if we
don't use CompositeType internally, for compatibility sake we need to style
parse and return stuffs like "CompositeType(UTF8Type, IntegerType)" in thrift
CfDef and such. And more generally, there is a few of places in the code where
it's just simpler to just convert the CType to an equivalent old style
CompositeType (the cql2 has some of those for instance). Another more minor
reason is that if someone is crazy enough to use a super column with a
compositeType comparator or subcomparator today, we'll still use the old
CompositeType. As for AbstractCompositeType specifically, it's also reused for
DynamicCompositeType.
That being said, I think that if we do the follow up ticket I mention above of
using CType for composite row keys to, then I think we can remove some bits
like the CompositeType.Builder.
bq. Why are you using the word 'Dense' internally when CQL3 uses 'Compact'?
Because they are not equivalent :). I'll admit, this could just be a bad choice
of terms, but I haven't found much better. But anyway, internally there is 2
dimensions to the layouts used distinguished by CQL3 which are:
# whether the comparator is either composite or not. In the code I use
isPacked() to mean "not composite" at times, though that's also a "I haven't
found better" term.
# whether one of the component (the last one or 2nd to last if there is a
collection) of the comparator stores the CQL3 column name. If that's the case,
then the CF is not 'dense' (I might use sparse at times in that case),
otherwise it is. Another more pragramatic to see that is to say that dense CFs
are those where 1 CQL3 row == 1 internal cell.
Those are the distinction that are useful internally.
But while the 4 combinations of those 2 dimensions make sense, only one of them
(non packed, non dense) is the default "CQL3 layout". All the others are
created in CQL3 using "WITH COMPACT STORAGE". And so in particular, a COMPACT
static CF, i.e. something like
{noformat}
CREATE TABLE foo {
k int PRIMARY KEY,
c1 int,
c2 text,
c3 double
} WITH COMPACT STORAGE
{noformat}
is *not* dense (but it is "packed"). One CQL3 row correspond to a group of
internal cells.
I'm happy to change the term 'dense' internally if someone has a better
suggestion, but using 'compact' would confuse things more than anything imo.
> Push composites support in the storage engine
> ---------------------------------------------
>
> Key: CASSANDRA-5417
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5417
> Project: Cassandra
> Issue Type: Improvement
> Reporter: Sylvain Lebresne
> Assignee: Sylvain Lebresne
> Fix For: 2.0
>
>
> CompositeType happens to be very useful and is now widely used: CQL3 heavily
> rely on it, and super columns are now using it too internally. Besides,
> CompositeType has been advised as a replacement of super columns on the
> thrift side for a while, so it's safe to assume that it's generally used
> there too.
> CompositeType has initially been introduced as just another AbstractType.
> Meaning that the storage engine has no nothing whatsoever of composites
> being, well, composite. This has the following drawbacks:
> * Because internally a composite value is handled as just a ByteBuffer, we
> end up doing a lot of extra work. Typically, each time we compare 2 composite
> value, we end up "deserializing" the components (which, while it doesn't copy
> data per-se because we just slice the global ByteBuffer, still waste some cpu
> cycles and allocate a bunch of ByteBuffer objects). And since compare can be
> called *a lot*, this is likely not negligible.
> * This make CQL3 code uglier than necessary. Basically, CQL3 makes extensive
> use of composites, and since it gets backs ByteBuffer from the internal
> columns, it always have to check if it's actually a compositeType or not, and
> then split it and pick the different parts it needs. It's only an API
> problem, but having things exposed as composites directly would definitively
> make thinks cleaner. In particular, in most cases, CQL3 don't care whether it
> has a composite with only one component or a non-really-composite value, but
> we still always distinguishes both cases. Lastly, if we do expose composites
> more directly internally, it's not a lot more work to "internalize" better
> the different parts of the cell name that CQL3 uses (what's the clustering
> key, what's the actuall CQL3 column name, what's the collection element),
> making things cleaner. Last but not least, there is currently a bunch of
> places where methods take a ByteBuffer as argument and it's hard to know
> whether it expects a cell name or a CQL3 column name. This is pretty error
> prone.
> * It makes it hard (or impossible) to do a number of performance
> improvements. Consider CASSANDRA-4175, I'm not really sure how you can do it
> properly (in memory) if cell names are just ByteBuffer (since CQL3 column
> names are just one of the component in general). But we also miss
> oportunities of sharing prefixes. If we were able to share prefixes of
> composite names in memory we would 1) lower the memory footprint and 2)
> potentially speed-up comparison (of the prefixes) by checking reference
> equality first (also, doing prefix sharing on-disk, which is a separate
> concern btw, might be easier to do if we do prefix sharing in memory).
> So I suggest pushing CompositeType support inside the storage engine. What I
> mean by that concretely would be change the internal {{Column.name}} from
> ByteBuffer to some CellName type. A CellName would API-wise just be a list of
> ByteBuffer. But in practice, we'd have a specific CellName implementation for
> not-really-composite names, and the truly composite implementation will allow
> some prefix sharing. From an external API however, nothing would change, we
> would pack the composite as usual before sending it back to the client, but
> at least internally, comparison won't have to deserialize the components
> every time, and CQL3 code will be cleaner.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
