HI Benchao, Thank you very much for your excellent suggestions! I believe that I have a better understanding about my problem now. I saw the pushdowns and also the usage of multiple conventions in the same plan, and was able to implement and play around with the same.
I went through talk [4] as well. So my question is better phrased as: "Is there a planner rule that converts a join on 2 Table scans to some kind of table?" Are any examples available on this? I tried looking for examples on the same, however; I was able to observe pushdowns, but not what I just mentioned. This is similar to the ProjectTableScan rule in some sense, it converts a project on a TableScan to a ProjectableFilterableTable SCAN. Thanks & Regards, Pranav On Fri, Aug 5, 2022 at 11:57 PM Benchao Li <libenc...@apache.org> wrote: > Pranav, > > You can reference Calcite adaptors implementation, such as JDBC > Adaptor[1], MongoDB Adaptor[2]. > Their implementation allows pushing down operations (RelNode) to the > adapter as much as possible, and the left RelNodes will be implemented > using Enumerable Convention. > > We have a Converter[3] concept which makes this possible. The converter > node allows multiple Conventions in a single query. This also answers your > second question. > There is a talk[4] about this, it's very helpful for understanding this > concept. > > [1] > https://github.com/apache/calcite/blob/main/core/src/main/java/org/apache/calcite/adapter/jdbc/JdbcToEnumerableConverter.java > [2] > https://github.com/apache/calcite/blob/main/mongodb/src/main/java/org/apache/calcite/adapter/mongodb/MongoToEnumerableConverter.java > [3] > https://github.com/apache/calcite/blob/main/core/src/main/java/org/apache/calcite/rel/convert/Converter.java > [4] > https://calcite.apache.org/community/#fast-federated-sql-with-apache-calcite > > Julian Hyde <jhyde.apa...@gmail.com> 于2022年8月6日周六 02:06写道： > >> Pranav, >> >> Please subscribe to this list. You have asked several questions, received >> replies, not acknowledged those replies, and asked further questions. Also, >> since you are not subscribed, each email you post has to go through manual >> moderation. >> >> Julian >> >> > On Aug 5, 2022, at 9:38 AM, Pranav Deshpande < >> deshpande.v.pra...@gmail.com> wrote: >> > >> > Dear Apache Calcite Team, >> > I have 2 questions. >> > >> > --------------------------------- >> > 1. >> > >> > There are plenty of examples on how to push down projects and filters >> into >> > the leaf nodes (tablescans). >> > >> > However, I could not find any examples to push down joins to TableScans >> (or >> > joins+filters+projects etc.) [this is helpful for data federation I >> think]. >> > >> > On the mailing list, many folks are suggesting that I use Drill. >> However, >> > the purpose of my exercise is to gain knowledge about DBMS and Query >> > processing etc. >> > >> > I tried debugging open source engines that use Calcite (Drill, Druid, >> Trino >> > etc.) but was completely lost. >> > >> > Any examples/pointers/guidance around the same would be appreciated. >> > Example, pushing down a join with a filter to a DBMS(consider jdbc >> > msql etc.) >> > >> > ------------------------------------- >> > 2. >> > >> > The 2nd question I have is regarding conventions and different DBMS. The >> > cluster has a method to replace the trait convention(Bindable,JDBC >> etc.), >> > and then we optimize and get the physical plan. >> > >> > But imagine I have both the MYSQL JDBC convnction and a cassandra >> > convention and some user is trying to query both tables. >> > >> > Something like "SELECT users.username, specialdata.country from >> > cassandraDB.user join mysqlDB.specialdata ON users.id >> =specialdata.userid" >> > >> > Now, how will calcite do the optimization here? The planner is not >> > accepting 2 different conventions. >> > >> > Thanks & Regards, >> > Pranav >> >> > > -- > > Best, > Benchao Li >
