+1 what Stamatis said. Queryable is for compatibility with LINQ. If you want to build an adapter that supports push-down, you will likely use FilterableTable for simple adapters, TranslatableTable for more complex adapters. In neither case will you need to deal with Queryable.
Stamatis laid out best practices in his excellent BOSS tutorial: https://www.youtube.com/watch?v=meI0W12f_nw <https://www.youtube.com/watch?v=meI0W12f_nw>. (I am co-presenter in other parts of the tutorial, but all credit for the adapters material goes to Stamatis.) Julian > On Jan 23, 2022, at 12:46 PM, Stamatis Zampetakis <zabe...@gmail.com> wrote: > > Hi Julian, > > I don't think there is an easy way to understand the Queryable interface > unless you check how it is used. I don't see it as something that you need > to implement no matter what but more as a convenience API that will > facilitate the integration with the Enumerable operators (if you rely on > them). Even in that case it could be possible to skip it. > > There are many ways to push a filter down into the underlying engine and I > think the Calcite code base has already quite a few examples on how this > can be done (JdbcAdapter, ElasticSearch, Druid, etc). There is no one > option that is best in all cases. Using one you may (e.g., FilterableTable) > write more concise code and using another (e.g., custom rules + custom > operators) may lead to more powerful optimizations or an easier extensible > system. > > Regarding materialized views there are many built in things in Calcite. The > best place to start would probably be the website [1] > > The bindable convention uses interpretation most of the time but it also > involves code generation in some parts (e.g., BindableFilter). The > Enumerable convention is more widely used in the wild so I would say it is > a more stable and better option to begin with. Afterwards you may need to > invest in building in-house operators to solve some invonveniences of > Calcite built-in conventions. > > Best, > Stamatis > > [1] https://calcite.apache.org/docs/materialized_views.html > > On Fri, Jan 21, 2022 at 1:21 PM Julian Feinauer < > j.feina...@pragmaticminds.de> wrote: > >> Hi all, >> >> in the last weeks I worked on Integrating the Apache IoTDB Project with >> Calcite. >> This covers two possible scenarios. One, to use Apache IoTDB as an Adapter >> in Apache Calcite (like MongoDB, Cassandra, et al) and on the other hand we >> are looking at using Calcites Query Optimizer to introduce indexing into >> the IoTDB server (the IoTDB Server builds a RelNode / Tree and passes it to >> the planner, after planning the resulting RelNode is then processed further >> by the IoTDB Server, executed and returned). >> >> I looked a lot on the other Adapters and how they are implemented and have >> some questions: >> >> One rather general question is about the Queryable<> Interface. I tried to >> look up all the docs (also in Linq) but still not fully understand it. From >> my understanding it is like a Enumerable<> but it has a “native” way to >> already to things like ordering or filtering. So if I have a Queryable<> >> which implements a custom Filter an automated “Push Down” can be done by >> the framework without a Rule or code generation. >> >> One important requirement for us in IoTDB is to do the query pushdown to >> the TableScan (which is done implicitly in the current server but is first >> explicit in the RelNode that we generate). >> So whats the best way to “merge” a LogicalFilter and a IoTDBTableScan to a >> “filtered” scan? >> Is the right way to return a QueryableTable as TableScan and the Planner >> will take care by generating the call to ‘.filter(…)’. >> The same applies to ordering. >> >> Another question that is important for us is the usage of “Materialized >> Views” or other “Indexes”. >> As we handle basically always timeseries in most cases the only suitable >> index is a “Materialized View” on parts of the time series which we can use >> to replace parts of the Relational Tree to avoid IO and computation for >> parts that are already precomputed. >> >> Is there already an existing support for that in Calcite or would we just >> write custom Rules for our cases? >> >> My last question is about the Callable TraitDef. So far I only used >> Enumerable Convention which results in Code generation (which has an >> impact on the query latency). Am I right in assuming that the Binable >> Convention is somehow similar to the Enumerable Convention with the only >> difference that it does not do code generation but interpretation? >> And to potentially use both (depending on whatever switch we set) we just >> have to provide Converter Rules for both? >> What would you use in a Server setup? Always Enumerable? >> >> Thanks already for any responses or hints! >> Julian F >>
