Thanks Stamatis, that makes sense. I'll keep using it as shown in those examples then.
Best, Nicola Il giorno lun 7 feb 2022 alle ore 22:52 Stamatis Zampetakis < [email protected]> ha scritto: > Hey Nicola, > > If you need a way to combine operators from the Enumerable convention, > which generate Java code, with other kind operators you need to have some > common interfaces to pass from one to the other. > > The XToEnumerableConverter needs to know how to generate code calling the > operators of the other convention thus using existing interfaces e.g., > Queryable comes in handy. Possibly it would be possible to combine > Enumerable with other conventions without using the interfaces you > mentioned but it would require more effort/code. > > Best, > Stamatis > > On Mon, Feb 7, 2022 at 11:17 PM Nicola Vitucci <[email protected]> > wrote: > > > I've seen that in many adapters, although they implement the > > TranslatableTable interface, Queryable is still used (see the usage of > > table.getExpression(...) in XToEnumerableConverter for Elasticsearch, > > Mongo, and Geode, to name a few). What is the reasoning there? > > > > Il giorno mar 25 gen 2022 alle ore 20:15 Julian Hyde < > > [email protected]> > > ha scritto: > > > > > +1 what Stamatis said. Queryable is for compatibility with LINQ. If you > > > want to build an adapter that supports push-down, you will likely use > > > FilterableTable for simple adapters, TranslatableTable for more complex > > > adapters. In neither case will you need to deal with Queryable. > > > > > > Stamatis laid out best practices in his excellent BOSS tutorial: > > > https://www.youtube.com/watch?v=meI0W12f_nw < > > > https://www.youtube.com/watch?v=meI0W12f_nw>. (I am co-presenter in > > other > > > parts of the tutorial, but all credit for the adapters material goes to > > > Stamatis.) > > > > > > Julian > > > > > > > > > > > > > On Jan 23, 2022, at 12:46 PM, Stamatis Zampetakis <[email protected] > > > > > wrote: > > > > > > > > Hi Julian, > > > > > > > > I don't think there is an easy way to understand the Queryable > > interface > > > > unless you check how it is used. I don't see it as something that you > > > need > > > > to implement no matter what but more as a convenience API that will > > > > facilitate the integration with the Enumerable operators (if you rely > > on > > > > them). Even in that case it could be possible to skip it. > > > > > > > > There are many ways to push a filter down into the underlying engine > > and > > > I > > > > think the Calcite code base has already quite a few examples on how > > this > > > > can be done (JdbcAdapter, ElasticSearch, Druid, etc). There is no one > > > > option that is best in all cases. Using one you may (e.g., > > > FilterableTable) > > > > write more concise code and using another (e.g., custom rules + > custom > > > > operators) may lead to more powerful optimizations or an easier > > > extensible > > > > system. > > > > > > > > Regarding materialized views there are many built in things in > Calcite. > > > The > > > > best place to start would probably be the website [1] > > > > > > > > The bindable convention uses interpretation most of the time but it > > also > > > > involves code generation in some parts (e.g., BindableFilter). The > > > > Enumerable convention is more widely used in the wild so I would say > it > > > is > > > > a more stable and better option to begin with. Afterwards you may > need > > to > > > > invest in building in-house operators to solve some invonveniences of > > > > Calcite built-in conventions. > > > > > > > > Best, > > > > Stamatis > > > > > > > > [1] https://calcite.apache.org/docs/materialized_views.html > > > > > > > > On Fri, Jan 21, 2022 at 1:21 PM Julian Feinauer < > > > > [email protected]> wrote: > > > > > > > >> Hi all, > > > >> > > > >> in the last weeks I worked on Integrating the Apache IoTDB Project > > with > > > >> Calcite. > > > >> This covers two possible scenarios. One, to use Apache IoTDB as an > > > Adapter > > > >> in Apache Calcite (like MongoDB, Cassandra, et al) and on the other > > > hand we > > > >> are looking at using Calcites Query Optimizer to introduce indexing > > into > > > >> the IoTDB server (the IoTDB Server builds a RelNode / Tree and > passes > > > it to > > > >> the planner, after planning the resulting RelNode is then processed > > > further > > > >> by the IoTDB Server, executed and returned). > > > >> > > > >> I looked a lot on the other Adapters and how they are implemented > and > > > have > > > >> some questions: > > > >> > > > >> One rather general question is about the Queryable<> Interface. I > > tried > > > to > > > >> look up all the docs (also in Linq) but still not fully understand > it. > > > From > > > >> my understanding it is like a Enumerable<> but it has a “native” way > > to > > > >> already to things like ordering or filtering. So if I have a > > Queryable<> > > > >> which implements a custom Filter an automated “Push Down” can be > done > > by > > > >> the framework without a Rule or code generation. > > > >> > > > >> One important requirement for us in IoTDB is to do the query > pushdown > > to > > > >> the TableScan (which is done implicitly in the current server but is > > > first > > > >> explicit in the RelNode that we generate). > > > >> So whats the best way to “merge” a LogicalFilter and a > IoTDBTableScan > > > to a > > > >> “filtered” scan? > > > >> Is the right way to return a QueryableTable as TableScan and the > > Planner > > > >> will take care by generating the call to ‘.filter(…)’. > > > >> The same applies to ordering. > > > >> > > > >> Another question that is important for us is the usage of > > “Materialized > > > >> Views” or other “Indexes”. > > > >> As we handle basically always timeseries in most cases the only > > suitable > > > >> index is a “Materialized View” on parts of the time series which we > > can > > > use > > > >> to replace parts of the Relational Tree to avoid IO and computation > > for > > > >> parts that are already precomputed. > > > >> > > > >> Is there already an existing support for that in Calcite or would we > > > just > > > >> write custom Rules for our cases? > > > >> > > > >> My last question is about the Callable TraitDef. So far I only used > > > >> Enumerable Convention which results in Code generation (which has > an > > > >> impact on the query latency). Am I right in assuming that the > Binable > > > >> Convention is somehow similar to the Enumerable Convention with the > > only > > > >> difference that it does not do code generation but interpretation? > > > >> And to potentially use both (depending on whatever switch we set) we > > > just > > > >> have to provide Converter Rules for both? > > > >> What would you use in a Server setup? Always Enumerable? > > > >> > > > >> Thanks already for any responses or hints! > > > >> Julian F > > > >> > > > > > > > > >
