Re: Using Calcite in a multi-tenant application
First, remember that the Calcite "interface Schema" is quite low level. It is basically a namespace that contains a set of tables and a set of sub-schemas. You can, and probably should, make it immutable. A Schema instance doesn’t know its own name or path. That allows you to “mount” it (as you would mount a filesystem) under several paths. This gives you a clue for how you could serve multiple tenants, each possibly seeing the same schema and tables (or possibly seeing different subsets). Just set a different root schema for each tenant’s connection. I believe that’s analogous to how a container like Docker works. Calcite does not, by default, write anything to disk. The built-in adapters read from foreign wrappers (e.g. connecting to another JDBC database or Cassandra or MongoDB or CSV files) but then process data in memory. If you wanted to, you could write a merge sort algorithm that uses sort runs on disk. But that’s not in Calcite currently. Julian > On Oct 19, 2022, at 8:19 PM, JiaTao Tao wrote: > > In fact, you can change calcite to use like HMS to manager schema > > Regards! > > Aron Tao > > > Mike Albritton 于2022年10月20日周四 07:16写道： > >> Hi all, >> >> >> I am considering using Calcite in a multi-tenant application that can run >> queries from different customers in the same process and have a few >> questions for those familiar with the code about this approach. >> >> >> Specifically, I would like to know if metadata is completely separated >> per-schema that is loaded or if it is ever shared between schemas? Also, I >> read in the docs somewhere that Calcite manages memory for cross schema >> joins which seemed to suggest it may persist data to disk temporarily. Is >> this the case or is there somewhere I can learn more about how data >> persistence works? Are there any other concerns that I should have when >> running Calcite in a multi-tenant application? >> >> >> Finally, if you or someone you know may be interested in a short term >> contract to assist with a Calcite POC, let me know. >> >> >> Regards, >> >> Mike >>
Re: RelOptRulesTest.java and RelOptRulesTest.xml
Do you think that RelOptRulesTest should fail if the resources are not sorted alphabetically? (I thought it did that at some point… I may be wrong. Anyway, I am a fan of ‘fail fast’. The person who introduces the error should be the person to fix it.) > On Oct 20, 2022, at 9:20 AM, Alessandro Solimando > wrote: > > Hello everyone, > I have recently added tests to RelOptRulesTest.java, took the chance to > update (maven instruction with gradle instructions, path has changed) and > improved them a little bit. > > While testing the commands I have noticed that few PRs in the last months > had tests added to RelOptRulesTest.java with a manual update of > RelOptRulesTest.xml (possibly due to the stale instructions). > > Manual updates often suffer from formatting issues, stale/wrong SQL > statements, different test order. > > Note that those discrepancies are not caught by running tests/CI, but it's > nonetheless problematic, especially because it forbids folks down the line > to re-generate the file automatically, because their PRs would carry the > changes from past work. > > In https://github.com/apache/calcite/pull/2944 I will put back on track > RelOptRulesTest.xml, please try to follow the procedure in the javadoc of > RelOptRulesTest.java and let me know if we can improve them even further in > your opinion. > > Best regards, > Alessandro
RelOptRulesTest.java and RelOptRulesTest.xml
Hello everyone, I have recently added tests to RelOptRulesTest.java, took the chance to update (maven instruction with gradle instructions, path has changed) and improved them a little bit. While testing the commands I have noticed that few PRs in the last months had tests added to RelOptRulesTest.java with a manual update of RelOptRulesTest.xml (possibly due to the stale instructions). Manual updates often suffer from formatting issues, stale/wrong SQL statements, different test order. Note that those discrepancies are not caught by running tests/CI, but it's nonetheless problematic, especially because it forbids folks down the line to re-generate the file automatically, because their PRs would carry the changes from past work. In https://github.com/apache/calcite/pull/2944 I will put back on track RelOptRulesTest.xml, please try to follow the procedure in the javadoc of RelOptRulesTest.java and let me know if we can improve them even further in your opinion. Best regards, Alessandro
