Re: CassandraAdapter (Add Type) and WHERE statement.
You're right that there are several types which are not supported by the Cassandra adapter. We would happily accept pull requests to add support for new types. You're also correct that Cassandra cannot efficiently execute queries which do not specify the partition key. Calcite will make those queries more efficient, but it can make it easier to execute queries that CQL does not directly support. Ultimately data is still stored based on the partition key, so if your query does not specify a partition key, Calcite will still need to issue an expensive cross-partition query to Cassandra. -- Michael Mior mm...@apache.org Le mer. 16 oct. 2019 à 07:57, Yanna elina a écrit : > > Hi guys , > > I study Calcite the benefits that a Cassandra-Calcite Adapter can bring , > as for example brings the possibility of join. > > the problem type defined into CassandraSchema.getRelDataType(..) is very > limited > some important type are missing boolean / array ect... > > I thought inherited from the class CassandraSchema for Override this > method and add more type but this method is used inside CassandraTable too. > > i would like to avoid to re-write fully this adapter :) > > do you have suggestions? > > My second question is : Cassandra is not optimized to have WHERE on key > not defined on cluster/partition key. I was wondering if calcite could play > a role without this mechanism to improve performance > > > Thank ! > > Yanna
Re: CassandraAdapter (Add Type) and WHERE statement.
Thank for reply Michael. yes i understood this on the documentation for example with "WHERE" statement calcite i force the . "ALLOW FILTERING; " and this can be expensive. I think there may be an interesting approach using STREAM. for example maintain a regular update between a cassandra TABLE and a STREAM TABLE. CASSANDRA_TABLE_A .(SELECT * FROM TABLE_A) > STREAM_TABLE_A . SELECT STREAM * FROM STREAM_TABLE_A WHERE username = 'JmuhsAaMdw' i guess it will be more efficient to directly make the WHERE from the STREAM than the cassandra_adapter using "allow filtering" a synchronization strategy can be set up between the cassandra table and the STREAM table what is your opinion about this approach ? Thanks ! Yana Le mer. 16 oct. 2019 à 17:08, Michael Mior a écrit : > You're right that there are several types which are not supported by > the Cassandra adapter. We would happily accept pull requests to add > support for new types. > > You're also correct that Cassandra cannot efficiently execute queries > which do not specify the partition key. Calcite will make those > queries more efficient, but it can make it easier to execute queries > that CQL does not directly support. Ultimately data is still stored > based on the partition key, so if your query does not specify a > partition key, Calcite will still need to issue an expensive > cross-partition query to Cassandra. > -- > Michael Mior > mm...@apache.org > > Le mer. 16 oct. 2019 à 07:57, Yanna elina a > écrit : > > > > Hi guys , > > > > I study Calcite the benefits that a Cassandra-Calcite Adapter can bring , > > as for example brings the possibility of join. > > > > the problem type defined into CassandraSchema.getRelDataType(..) is very > > limited > > some important type are missing boolean / array ect... > > > > I thought inherited from the class CassandraSchema for Override this > > method and add more type but this method is used inside CassandraTable > too. > > > > i would like to avoid to re-write fully this adapter :) > > > > do you have suggestions? > > > > My second question is : Cassandra is not optimized to have WHERE on key > > not defined on cluster/partition key. I was wondering if calcite could > play > > a role without this mechanism to improve performance > > > > > > Thank ! > > > > Yanna >
Re: CassandraAdapter (Add Type) and WHERE statement.
Perhaps I'm missing something, but I don't see why this would be any more efficient. Selecting all data is also not an efficient operation in Cassandra. Using ALLOW FILTERING will likely be more efficient since it's basically the same as doing a table scan, but it avoids returning data which would later be filtered by Calcite anyway. -- Michael Mior mm...@apache.org Le jeu. 17 oct. 2019 à 09:13, Yanna elina a écrit : > > Thank for reply Michael. > > yes i understood this on the documentation for example with "WHERE" > statement calcite i force the . "ALLOW FILTERING; " > and this can be expensive. > > I think there may be an interesting approach using STREAM. > > for example maintain a regular update between a cassandra TABLE and a > STREAM TABLE. > > CASSANDRA_TABLE_A .(SELECT * FROM TABLE_A) > STREAM_TABLE_A . > SELECT STREAM * FROM STREAM_TABLE_A WHERE username = 'JmuhsAaMdw' > > i guess it will be more efficient to directly make the WHERE from the > STREAM than the cassandra_adapter using "allow filtering" > a synchronization strategy can be set up between the cassandra table and > the STREAM table > what is your opinion about this approach ? > Thanks ! > Yana > > > Le mer. 16 oct. 2019 à 17:08, Michael Mior a écrit : > > > You're right that there are several types which are not supported by > > the Cassandra adapter. We would happily accept pull requests to add > > support for new types. > > > > You're also correct that Cassandra cannot efficiently execute queries > > which do not specify the partition key. Calcite will make those > > queries more efficient, but it can make it easier to execute queries > > that CQL does not directly support. Ultimately data is still stored > > based on the partition key, so if your query does not specify a > > partition key, Calcite will still need to issue an expensive > > cross-partition query to Cassandra. > > -- > > Michael Mior > > mm...@apache.org > > > > Le mer. 16 oct. 2019 à 07:57, Yanna elina a > > écrit : > > > > > > Hi guys , > > > > > > I study Calcite the benefits that a Cassandra-Calcite Adapter can bring , > > > as for example brings the possibility of join. > > > > > > the problem type defined into CassandraSchema.getRelDataType(..) is very > > > limited > > > some important type are missing boolean / array ect... > > > > > > I thought inherited from the class CassandraSchema for Override this > > > method and add more type but this method is used inside CassandraTable > > too. > > > > > > i would like to avoid to re-write fully this adapter :) > > > > > > do you have suggestions? > > > > > > My second question is : Cassandra is not optimized to have WHERE on key > > > not defined on cluster/partition key. I was wondering if calcite could > > play > > > a role without this mechanism to improve performance > > > > > > > > > Thank ! > > > > > > Yanna > >
Re: CassandraAdapter (Add Type) and WHERE statement.
I think it's me who does not have to understand all the subtlety. I thought that STREAM works more like a in-memory- relational database but i missed something thank for your help :) Le jeu. 17 oct. 2019 à 15:53, Michael Mior a écrit : > Perhaps I'm missing something, but I don't see why this would be any > more efficient. Selecting all data is also not an efficient operation > in Cassandra. Using ALLOW FILTERING will likely be more efficient > since it's basically the same as doing a table scan, but it avoids > returning data which would later be filtered by Calcite anyway. > -- > Michael Mior > mm...@apache.org > > Le jeu. 17 oct. 2019 à 09:13, Yanna elina a > écrit : > > > > Thank for reply Michael. > > > > yes i understood this on the documentation for example with "WHERE" > > statement calcite i force the . "ALLOW FILTERING; " > > and this can be expensive. > > > > I think there may be an interesting approach using STREAM. > > > > for example maintain a regular update between a cassandra TABLE and a > > STREAM TABLE. > > > > CASSANDRA_TABLE_A .(SELECT * FROM TABLE_A) > STREAM_TABLE_A . > > SELECT STREAM * FROM STREAM_TABLE_A WHERE username = 'JmuhsAaMdw' > > > > i guess it will be more efficient to directly make the WHERE from the > > STREAM than the cassandra_adapter using "allow filtering" > > a synchronization strategy can be set up between the cassandra table and > > the STREAM table > > what is your opinion about this approach ? > > Thanks ! > > Yana > > > > > > Le mer. 16 oct. 2019 à 17:08, Michael Mior a écrit : > > > > > You're right that there are several types which are not supported by > > > the Cassandra adapter. We would happily accept pull requests to add > > > support for new types. > > > > > > You're also correct that Cassandra cannot efficiently execute queries > > > which do not specify the partition key. Calcite will make those > > > queries more efficient, but it can make it easier to execute queries > > > that CQL does not directly support. Ultimately data is still stored > > > based on the partition key, so if your query does not specify a > > > partition key, Calcite will still need to issue an expensive > > > cross-partition query to Cassandra. > > > -- > > > Michael Mior > > > mm...@apache.org > > > > > > Le mer. 16 oct. 2019 à 07:57, Yanna elina > a > > > écrit : > > > > > > > > Hi guys , > > > > > > > > I study Calcite the benefits that a Cassandra-Calcite Adapter can > bring , > > > > as for example brings the possibility of join. > > > > > > > > the problem type defined into CassandraSchema.getRelDataType(..) is > very > > > > limited > > > > some important type are missing boolean / array ect... > > > > > > > > I thought inherited from the class CassandraSchema for Override this > > > > method and add more type but this method is used inside > CassandraTable > > > too. > > > > > > > > i would like to avoid to re-write fully this adapter :) > > > > > > > > do you have suggestions? > > > > > > > > My second question is : Cassandra is not optimized to have WHERE on > key > > > > not defined on cluster/partition key. I was wondering if calcite > could > > > play > > > > a role without this mechanism to improve performance > > > > > > > > > > > > Thank ! > > > > > > > > Yanna > > > >
Re: CassandraAdapter (Add Type) and WHERE statement.
Hello, I have logged a Jira ticket for this: https://issues.apache.org/jira/browse/CALCITE-3465 I have listed all the data types for Cassandra 3.x, and I have tried to compile a table with the current status, but there are few entries for which I am not sure. If you have time to contribute to the table or material I can use to fill the blanks, it would be great. Best regards, Alessandro On Wed, 16 Oct 2019 at 17:08, Michael Mior wrote: > You're right that there are several types which are not supported by > the Cassandra adapter. We would happily accept pull requests to add > support for new types. > > You're also correct that Cassandra cannot efficiently execute queries > which do not specify the partition key. Calcite will make those > queries more efficient, but it can make it easier to execute queries > that CQL does not directly support. Ultimately data is still stored > based on the partition key, so if your query does not specify a > partition key, Calcite will still need to issue an expensive > cross-partition query to Cassandra. > -- > Michael Mior > mm...@apache.org > > Le mer. 16 oct. 2019 à 07:57, Yanna elina a > écrit : > > > > Hi guys , > > > > I study Calcite the benefits that a Cassandra-Calcite Adapter can bring , > > as for example brings the possibility of join. > > > > the problem type defined into CassandraSchema.getRelDataType(..) is very > > limited > > some important type are missing boolean / array ect... > > > > I thought inherited from the class CassandraSchema for Override this > > method and add more type but this method is used inside CassandraTable > too. > > > > i would like to avoid to re-write fully this adapter :) > > > > do you have suggestions? > > > > My second question is : Cassandra is not optimized to have WHERE on key > > not defined on cluster/partition key. I was wondering if calcite could > play > > a role without this mechanism to improve performance > > > > > > Thank ! > > > > Yanna >
