> If we provide a way to drop the flag, but still access the data, I think that > is fine and perfectly reasonable. If the proposal here is that users who > have data in COMPACT STORAGE tables have no way to upgrade to 4.0 and still > access that data without exporting it to a brand new table, then I am against > it. Can you clarify which thing is being proposed? It is not clear to me.
A bit of details on how compact storage and all thrift tables are implemented: When a table is created through thrift or with COMPACT STORAGE flag, it has a `value` column, which is invisible when doing any CQL queries and only seen through Thrift. With SuperColumn families, internally (on the storage level) have a partition key, clustering and a value column that has a type of `map<>`. Thrift exposes it as a “normal” super column family. CQL, however, does not expose this `map` column. Instead, it translates the key of the map into the second clustering and a map value as a regular column. All of this requires quite some special-casing everywhere across the CQL layer, in order to hide/show and translate the columns depending on whether the table is dense, super and so on. For more details you can take a look at 8099 or 12373. In short: dropping a COMPACT STORAGE flag means that your tables will be accessible and their internal representation (e.g. hidden value column) will be exposed as if it was a normal column. No data will be lost, no data will be inaccessible. You can take a look at the details of CASSANDRA-10857 if you want more details. As regards SuperColumn families, my proposal is to have a 100% support in 3.0/3.11 (LWTs, counters, all sorts of queries, exactly like they were accessible through CQL in 2.2). There will be a clear upgrade path, but I suggest that the DROP COMPACT STORAGE has to be in 3.x only. 4.x will still make the same data available, but expose the whole internal CQL structure, together with a usually “hidden" compact value column, without any legacy special-casing. > On 19 Sep 2017, at 17:23, Jeremiah D Jordan <jeremiah.jor...@gmail.com> wrote: > > I think that all the work to support Compact Storage tables from CQL seems > like wasted effort if we are going to tell people “just kidding, you have to > migrate all your data”. I do not think supporting “COMPACT STORAGE” as a > table option matters one way or the other. But I do think being able to read > the data that was in a table created that way is something we need to have a > path forward for. > >> since thrift is not supported on trunk/4.0, it makes it much less appealing >> or even necessary > > I think that the fact thrift is not supported on trunk/4.0 makes accessing > said data from CQL *MORE* necessary and appealing. > >> possibility drop a Compact Storage flag and expose them as “normal" tables, >> there was an idea of removing the Compact Tables from 4.x altogether. > > If we provide a way to drop the flag, but still access the data, I think that > is fine and perfectly reasonable. If the proposal here is that users who > have data in COMPACT STORAGE tables have no way to upgrade to 4.0 and still > access that data without exporting it to a brand new table, then I am against > it. Can you clarify which thing is being proposed? It is not clear to me. > > -Jeremiah > > >> On Sep 19, 2017, at 7:10 AM, Oleksandr Petrov <oleksandr.pet...@gmail.com >> <mailto:oleksandr.pet...@gmail.com>> wrote: >> >> As you may know, SuperColumn Tables did not work in 3.x the way they worked >> in 2.x. In order to provide everyone with a reasonable upgrade path, we've >> been working on CASSANDRA-12373[1], that brings in support for SuperColumn >> tables as close to 2.x as possible. The patch is planned to land >> cassandra-3.0 and cassandra-3.11 branches only, since the patch for trunk >> will require even more work and, since thrift is not supported on trunk/4.0, >> it makes it much less appealing or even necessary. The idea behind the >> support for SuperColumns was always only to allow people to smoothly migrate >> off them in 3.0/3.11 world, not to have them as a primary feature. >> >> SuperColumns are not the only type of Compact Table, there are more. After >> CASSANDRA-8099[2], Compact Tables are special-cased and have special schema >> layout with some columns hidden from CQL, that allows them to be used from >> Thrift. But, except for the fact they’re accessible from Thrift, there are >> no advantages to use them with the new storage. In order to allow people to >> “expose” the internal structure of the compact tables to make them fully >> accessible in CQL, CASSANDRA-10857[3] was created. >> >> In the light of the fact that 4.0 will not have reasonable SuperColumn >> support (due to related complexity and amount of special-cases required to >> support it in 4.0) and a possibility drop a Compact Storage flag and expose >> them as “normal" tables, there was an idea of removing the Compact Tables >> from 4.x altogether. >> >> >> Leaving Compact Storage in 3.x only will make the table metadata a bit >> lighter and allow us to remove some special cases required for their >> support. Doing it during the major release, provided with a reasonable >> upgrade path (same functionality from both Thrift and CQL for all compact >> tables, including Super Column ones) through 3.x/3.11, sounds like the best >> option that we have right now. >> >> It’d be good if you could voice your support of this idea (or raise possible >> concerns, if there are any). >> >> >> There will be additional discussion and a proposal on how to allow “online” >> COMPACT STORAGE flag drop in CASSANDRA-10857 later this (or the following >> week). >> >> Best Regards, >> Alex >> >> [1] https://issues.apache.org/jira/browse/CASSANDRA-12373 >> <https://issues.apache.org/jira/browse/CASSANDRA-12373> >> <https://issues.apache.org/jira/browse/CASSANDRA-12373 >> <https://issues.apache.org/jira/browse/CASSANDRA-12373>> >> [2] https://issues.apache.org/jira/browse/CASSANDRA-8099 >> <https://issues.apache.org/jira/browse/CASSANDRA-8099> >> <https://issues.apache.org/jira/browse/CASSANDRA-8099 >> <https://issues.apache.org/jira/browse/CASSANDRA-8099>> >> [3] https://issues.apache.org/jira/browse/CASSANDRA-10857 >> <https://issues.apache.org/jira/browse/CASSANDRA-10857> >> <https://issues.apache.org/jira/browse/CASSANDRA-10857 >> <https://issues.apache.org/jira/browse/CASSANDRA-10857>> >> > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > <mailto:dev-unsubscr...@cassandra.apache.org> > For additional commands, e-mail: dev-h...@cassandra.apache.org > <mailto:dev-h...@cassandra.apache.org>
