4.0 should also fail startup (very early) if it still sees any non-migrated tables, probably.
— AY On 19 September 2017 at 18:35:11, J. D. Jordan (jeremiah.jor...@gmail.com) wrote: Thanks for the clarification. +1 for adding a "DROP COMPACT STORAGE" option in 3.x and then not allowing it to be specified in 4.0. On Sep 19, 2017, at 1:27 PM, Alex P <oleksandr.pet...@gmail.com> wrote: >> If we provide a way to drop the flag, but still access the data, I think >> that is fine and perfectly reasonable. If the proposal here is that users >> who have data in COMPACT STORAGE tables have no way to upgrade to 4.0 and >> still access that data without exporting it to a brand new table, then I am >> against it. Can you clarify which thing is being proposed? It is not clear >> to me. > > > A bit of details on how compact storage and all thrift tables are > implemented: > > When a table is created through thrift or with COMPACT STORAGE flag, it has a > `value` column, which is invisible when doing any CQL queries and only seen > through Thrift. > With SuperColumn families, internally (on the storage level) have a partition > key, clustering and a value column that has a type of `map<>`. Thrift exposes > it as a “normal” super column family. CQL, however, does not expose this > `map` column. Instead, it translates the key of the map into the second > clustering and a map value as a regular column. > > All of this requires quite some special-casing everywhere across the CQL > layer, in order to hide/show and translate the columns depending on whether > the table is dense, super and so on. > > > > For more details you can take a look at 8099 or 12373. > > In short: dropping a COMPACT STORAGE flag means that your tables will be > accessible and their internal representation (e.g. hidden value column) will > be exposed as if it was a normal column. No data will be lost, no data will > be inaccessible. You can take a look at the details of CASSANDRA-10857 if you > want more details. > > As regards SuperColumn families, my proposal is to have a 100% support in > 3.0/3.11 (LWTs, counters, all sorts of queries, exactly like they were > accessible through CQL in 2.2). > > There will be a clear upgrade path, but I suggest that the DROP COMPACT > STORAGE has to be in 3.x only. > > 4.x will still make the same data available, but expose the whole internal > CQL structure, together with a usually “hidden" compact value column, without > any legacy special-casing. > > > > > > > >> On 19 Sep 2017, at 17:23, Jeremiah D Jordan <jeremiah.jor...@gmail.com> >> wrote: >> >> I think that all the work to support Compact Storage tables from CQL seems >> like wasted effort if we are going to tell people “just kidding, you have to >> migrate all your data”. I do not think supporting “COMPACT STORAGE” as a >> table option matters one way or the other. But I do think being able to read >> the data that was in a table created that way is something we need to have a >> path forward for. >> >>> since thrift is not supported on trunk/4.0, it makes it much less appealing >>> or even necessary >> >> I think that the fact thrift is not supported on trunk/4.0 makes accessing >> said data from CQL *MORE* necessary and appealing. >> >>> possibility drop a Compact Storage flag and expose them as “normal" tables, >>> there was an idea of removing the Compact Tables from 4.x altogether. >> >> If we provide a way to drop the flag, but still access the data, I think >> that is fine and perfectly reasonable. If the proposal here is that users >> who have data in COMPACT STORAGE tables have no way to upgrade to 4.0 and >> still access that data without exporting it to a brand new table, then I am >> against it. Can you clarify which thing is being proposed? It is not clear >> to me. >> >> -Jeremiah >> >> >>> On Sep 19, 2017, at 7:10 AM, Oleksandr Petrov <oleksandr.pet...@gmail.com >>> <mailto:oleksandr.pet...@gmail.com>> wrote: >>> >>> As you may know, SuperColumn Tables did not work in 3.x the way they worked >>> in 2.x. In order to provide everyone with a reasonable upgrade path, we've >>> been working on CASSANDRA-12373[1], that brings in support for SuperColumn >>> tables as close to 2.x as possible. The patch is planned to land >>> cassandra-3.0 and cassandra-3.11 branches only, since the patch for trunk >>> will require even more work and, since thrift is not supported on >>> trunk/4.0, it makes it much less appealing or even necessary. The idea >>> behind the support for SuperColumns was always only to allow people to >>> smoothly migrate off them in 3.0/3.11 world, not to have them as a primary >>> feature. >>> >>> SuperColumns are not the only type of Compact Table, there are more. After >>> CASSANDRA-8099[2], Compact Tables are special-cased and have special schema >>> layout with some columns hidden from CQL, that allows them to be used from >>> Thrift. But, except for the fact they’re accessible from Thrift, there are >>> no advantages to use them with the new storage. In order to allow people to >>> “expose” the internal structure of the compact tables to make them fully >>> accessible in CQL, CASSANDRA-10857[3] was created. >>> >>> In the light of the fact that 4.0 will not have reasonable SuperColumn >>> support (due to related complexity and amount of special-cases required to >>> support it in 4.0) and a possibility drop a Compact Storage flag and expose >>> them as “normal" tables, there was an idea of removing the Compact Tables >>> from 4.x altogether. >>> >>> >>> Leaving Compact Storage in 3.x only will make the table metadata a bit >>> lighter and allow us to remove some special cases required for their >>> support. Doing it during the major release, provided with a reasonable >>> upgrade path (same functionality from both Thrift and CQL for all compact >>> tables, including Super Column ones) through 3.x/3.11, sounds like the best >>> option that we have right now. >>> >>> It’d be good if you could voice your support of this idea (or raise >>> possible concerns, if there are any). >>> >>> >>> There will be additional discussion and a proposal on how to allow “online” >>> COMPACT STORAGE flag drop in CASSANDRA-10857 later this (or the following >>> week). >>> >>> Best Regards, >>> Alex >>> >>> [1] https://issues.apache.org/jira/browse/CASSANDRA-12373 >>> <https://issues.apache.org/jira/browse/CASSANDRA-12373> >>> <https://issues.apache.org/jira/browse/CASSANDRA-12373 >>> <https://issues.apache.org/jira/browse/CASSANDRA-12373>> >>> [2] https://issues.apache.org/jira/browse/CASSANDRA-8099 >>> <https://issues.apache.org/jira/browse/CASSANDRA-8099> >>> <https://issues.apache.org/jira/browse/CASSANDRA-8099 >>> <https://issues.apache.org/jira/browse/CASSANDRA-8099>> >>> [3] https://issues.apache.org/jira/browse/CASSANDRA-10857 >>> <https://issues.apache.org/jira/browse/CASSANDRA-10857> >>> <https://issues.apache.org/jira/browse/CASSANDRA-10857 >>> <https://issues.apache.org/jira/browse/CASSANDRA-10857>> >>> >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org >> <mailto:dev-unsubscr...@cassandra.apache.org> >> For additional commands, e-mail: dev-h...@cassandra.apache.org >> <mailto:dev-h...@cassandra.apache.org> --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org For additional commands, e-mail: dev-h...@cassandra.apache.org
