> If you wanted to, you could write a merge sort algorithm that uses sort runs on disk. But that’s not in Calcite currently. Just to complete this info, there is a ticket logged for that feature [1], but AFAIK there is nobody working on it at the moment.
[1] https://issues.apache.org/jira/browse/CALCITE-4193 On Thu, Oct 20, 2022 at 9:01 PM Julian Hyde <jhyde.apa...@gmail.com> wrote: > First, remember that the Calcite "interface Schema" is quite low level. It > is basically a namespace that contains a set of tables and a set of > sub-schemas. You can, and probably should, make it immutable. > > A Schema instance doesn’t know its own name or path. That allows you to > “mount” it (as you would mount a filesystem) under several paths. > > This gives you a clue for how you could serve multiple tenants, each > possibly seeing the same schema and tables (or possibly seeing different > subsets). Just set a different root schema for each tenant’s connection. I > believe that’s analogous to how a container like Docker works. > > Calcite does not, by default, write anything to disk. The built-in > adapters read from foreign wrappers (e.g. connecting to another JDBC > database or Cassandra or MongoDB or CSV files) but then process data in > memory. > > If you wanted to, you could write a merge sort algorithm that uses sort > runs on disk. But that’s not in Calcite currently. > > Julian > > > > On Oct 19, 2022, at 8:19 PM, JiaTao Tao <taojia...@gmail.com> wrote: > > > > In fact, you can change calcite to use like HMS to manager schema > > > > Regards! > > > > Aron Tao > > > > > > Mike Albritton <mike.albrit...@gmail.com> 于2022年10月20日周四 07:16写道： > > > >> Hi all, > >> > >> > >> I am considering using Calcite in a multi-tenant application that can > run > >> queries from different customers in the same process and have a few > >> questions for those familiar with the code about this approach. > >> > >> > >> Specifically, I would like to know if metadata is completely separated > >> per-schema that is loaded or if it is ever shared between schemas? > Also, I > >> read in the docs somewhere that Calcite manages memory for cross schema > >> joins which seemed to suggest it may persist data to disk temporarily. > Is > >> this the case or is there somewhere I can learn more about how data > >> persistence works? Are there any other concerns that I should have when > >> running Calcite in a multi-tenant application? > >> > >> > >> Finally, if you or someone you know may be interested in a short term > >> contract to assist with a Calcite POC, let me know. > >> > >> > >> Regards, > >> > >> Mike > >> > >
