Thank you guys for the information, ill go over everything and keep you updated on the progress.
Stamatic, i currently use calcite as intended - to provide an execution system over my data storage; As a prototype, i'm now attempting to integrate with an *existing* software, such that some of the queries it receives are dispatched to my system; This should come in in the form of that system's Plan rather than an SQL query. I see this almost the "reverse" flow of what omnicsi attempted (As referenced by Michael) - taking an existing representation into calcite. Thanks again for the help! On Tue, Apr 7, 2020 at 7:51 PM Julian Hyde <jh...@apache.org> wrote: > Andrii, > > You mentioned that "optimizations that happen during SQL-to-Rel > conversion": > > > One thing to consider is there are some optimizations that happen during > > SQL-to-Rel conversion, so you won't get those if you use RelBuilder > > directly. > > I'd like to minimize these. Optimizations/simplifications should be > done by RelBuilder and/or by rewrite rules, and SQL-to-Rel should only > concern itself with the peculiarities of SQL. For example, we used to > do decorrelation and elimination of scalar sub-queries in > SqlToRelConverter; now these are done by rewrite rules, it is a big > improvement. > > Could you list the optimizations that are present in SqlToRelConverter > and not present elsewhere? > > I would be open to adding support for SQL-style operations such as > CASE, COALESCE, IN sub-query and IN value-list to RelBuilder. > > Julian > > > On Tue, Apr 7, 2020 at 1:30 AM Андрей Цвелодуб <a.tsvelo...@gmail.com> > wrote: > > > > Hi Tal, > > > > Although our case is not really similar to yours, we had to do a > conversion > > from our custom API to Calcite RelNodes. We just used RelBuilder to build > > the tree, and then optimized and executed it. > > I would say there is nothing really difficult, most of that logic could > be > > found in Calcite itself, so you can copy it and customize it to your > needs. > > > > One thing to consider is there are some optimizations that happen during > > SQL-to-Rel conversion, so you won't get those if you use RelBuilder > > directly. And there are some corner-cases like IN filters or > > implementations of some SQL functions that are also handled in > > SqlToRelConverter (e.g. CASE is used to implement COALESCE). Most of > those > > could be reimplemented, there's usually nothing really complex there. > > > > Best Regards, > > Andrii Tsvielodub > > > > On Sun, 5 Apr 2020 at 17:47, Stamatis Zampetakis <zabe...@gmail.com> > wrote: > > > > > Hi Tal, > > > > > > I'm sure there are valid reasons that you are using your own SQL > parser and > > > not the one provided by Calcite but can you briefly explain why? This > is to > > > understand if there are things that we could do to improve Calcite. > > > > > > I don't have any particular experience for the use case you mentioned > but > > > one thing that comes first to my mind is to serialize, send, and > > > deserialize the entire plan [1, 2, 3]. > > > > > > Best, > > > Stamatis > > > > > > [1] > > > > > > > https://github.com/apache/calcite/blob/d0180d160120e5d775b000549b7edac30250a353/core/src/test/java/org/apache/calcite/plan/RelWriterTest.java#L491 > > > [2] > > > > > > > https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/rel/externalize/RelJsonReader.java > > > [3] > > > > > > > https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/rel/externalize/RelJsonWriter.java > > > > > > On Sun, Apr 5, 2020, 11:44 AM Tal Glanzman <talglanz...@gmail.com> > wrote: > > > > > > > Hi, > > > > > > > > I'm looking for a some guidance / references on how to approach the > > > > following scenario. > > > > > > > > I have 2 separate software modules > > > > 1. Cpp-Codebase that parses an SQL query and construct a plan > > > > 2. Calcite adapter to my specific domain > > > > > > > > My goal is to provide a server, such that the Cpp-Codebase, as the > > > client, > > > > can construct (in the server) a corresponding plan (of calcite). The > > > client > > > > shouldn't receive the constructed plan back. > > > > > > > > Basically, it's an integration problem, i want to convert > > > NativePlan@Client > > > > to RelNode@Server. > > > > > > > > *I am looking for references based on your experience how would i go > > > about > > > > this. * > > > > > > > > My initial intuition is to provide a gRPC service, with messages that > > > > according to will construct the plan > > > > - the client will convert it's model to the gRPC model > > > > - the server will convert the gRPC model to the RelNode > > > > > > > > This also includes a manual construction of the RelNode which im not > sure > > > > how to go about... > > > > > > > > Thanks in advance! > > > > Tal > > > > > > > >
