Julian, sounds very interesting! > " I wonder whether we can remove the need to create those Foo-specific classes and rules. Could people just write a FooQuery relational expression that had a sequence of lightweight relational operations inside it?"
What might something like this look like, out of curiosity? On Wed, Jan 26, 2022 at 3:28 AM Nicola Vitucci <[email protected]> wrote: > Hi Julian, > > That would be excellent. It would help simplify the development quite a > bit, and reduce repetition too. Do you already have any ideas on how to do > it? > > Nicola > > > Il mar 25 gen 2022, 20:33 Julian Hyde <[email protected]> ha scritto: > > > This reminds me that there’s an opportunity to make it easier to write > > adapters. There is a continuum of backends for adapters, from simple > (such > > as CSV, which supports project and maybe filter) to powerful (such as > > Elasticsearch, which supports full relational algebra and then some). I > am > > thinking about the latter case. > > > > Let’s suppose we are writing an adapter for a database Foo which is > > approximately as powerful as SQL. Typically we write such adapters by > > creating a Foo convention, then a FooToEnumerableConverter, and then > write > > rules to push each relational operator down through the membrane. > > > > (Elasticsearch, Cassandra and Druid are all in this category, and all use > > the basic ‘membrane’ principle, but it is worrying that their adapters > are > > so dissimilar.) > > > > If Foo has ten relational operators, you need to write ten RelNode > classes > > and ten rules. That is a considerable amount of code, which ends up > looking > > like boilerplate. I wonder whether we can remove the need to create those > > Foo-specific classes and rules. Could people just write a FooQuery > > relational expression that had a sequence of lightweight relational > > operations inside it? > > > > Note that in MutableRel we have lightweight equivalents to RelNodes, for > > example MutableProject [1], and they could be repurposed. > > > > The adapter would still have some hard work to do: translating RexNode to > > their expression language (splitting or rejecting expressions that they > > can’t handle), figuring out which sequences of relational operations are > > valid, and translating the FooQuery. But if we reduce the boilerplate, > the > > adapter author can focus on the hard parts. > > > > Julian > > > > [1] > > > https://github.com/apache/calcite/blob/4bc916619fd286b2c0cc4d5c653c96a68801d74e/core/src/main/java/org/apache/calcite/rel/mutable/MutableProject.java#L34 > > > > > On Jan 23, 2022, at 12:28 PM, Stamatis Zampetakis <[email protected]> > > wrote: > > > > > > To the best of my knowledge there is no document getting into details > > about > > > the Queryable interface etc. The best way to approach this as Gavin > > > mentioned is to see how it is used in the project and outside of it. > > > > > > One thing that I wanted to mention is that you can build a system using > > > Calcite without using these interfaces at all. If you consider for > > instance > > > the druid module in Calcite I think it doesn't rely on Queryable at > all. > > > > > > Best, > > > Stamatis > > > > > > > > > On Wed, Jan 19, 2022 at 4:00 PM Gavin Ray <[email protected]> > wrote: > > > > > >> My apologies, I didn't realize that the updated search wasn't public > now > > >> (it opened for me a few days ago) > > >> > > >> Regular GH links: > > >> https://github.com/search?q=rawqueryable+language%3AJava&type=code > > >> > > >> Searching "CalciteConnection" across all repos, but removing > > >> "apache/calcite" from the results: > > >> > > >> > > > https://github.com/search?q=CalciteConnection+-repo%3Aapache%2Fcalcite&type=code > > >> > > >> > > >> > > >> On Tue, Jan 18, 2022 at 12:17 PM M Singh <[email protected] > > > > >> wrote: > > >> > > >>> Thanks Gavin for your pointers. I am in the waiting list for the > > search > > >>> tool you mentioned. I will continue to search on google in the > > >> meanwhile. > > >>> Mans > > >>> > > >>> > > >>> On Tuesday, January 18, 2022, 08:44:31 AM EST, Gavin Ray < > > >>> [email protected]> wrote: > > >>> > > >>> Not to be dismissive, but one thing I've done that has been helpful > for > > >>> me, > > >>> is to search all public repos for usage/examples of the thing I am > > trying > > >>> to understand. > > >>> For example, this "RawQueryable" does not appear to be utilized in > any > > >>> user-facing code on Github, so it may not be useful for you: > > >>> > > >>> > > >> > > > https://cs.github.com/?q=rawqueryable%20language%3AJava&scopeName=All%20repos&scope= > > >>> > > >>> However, take something like "CalciteConnection" or "RelNode", try to > > >>> remove the Calcite repo itself, and you'll get a lot of hits and > usage > > >>> examples: > > >>> > > >>> > > >> > > > https://cs.github.com/?scopeName=All+repos&scope=&q=CalciteConnection+%28NOT+path%3A%22calcite%2F%22%29 > > >>> > > >>> (I find that searching Scala repos usually gives more relevant > results, > > >> due > > >>> to lots of Java results being duplicates) > > >>> > > >>> This isn't nearly as good as getting a direct answer, but sometimes > you > > >>> might be able to find what you're looking for > > >>> > > >>> > > >>> > > >>> > > >>> On Tue, Jan 18, 2022 at 5:39 AM M Singh <[email protected] > > > > >>> wrote: > > >>> > > >>>> Hey Folks: > > >>>> Just wanted to see if you have any pointers to help me understand > the > > >>>> queryable concepts. > > >>>> Thanks > > >>>> On Friday, January 14, 2022, 09:09:06 AM EST, M Singh > > >>>> <[email protected]> wrote: > > >>>> > > >>>> Hi Folks: > > >>>> A few newbie questions - > > >>>> 1. I wanted to find out what are the benefits of a queryable table ( > > >>>> > > >>> > > >> > > > https://calcite.apache.org/javadocAggregate/org/apache/calcite/schema/QueryableTable.html > > >> ) > > >>> and > > >>>> what are the pros and cons of using it. > > >>>> 2. What is the role of Querable interface ( > > >>>> > > >>> > > >> > > > https://calcite.apache.org/javadocAggregate/org/apache/calcite/linq4j/Queryable.html > > >>> ) > > >>>> ?3. There is a getExpression methods in queryabletable ( > > >>>> > > >>> > > >> > > > https://calcite.apache.org/javadocAggregate/org/apache/calcite/linq4j/tree/Expression.html > > >>> ), > > >>>> and I wanted to find out if it is required and what are the > pros/cons > > >> of > > >>>> implementing it. > > >>>> 4. What role does RawQueralbe play ? ( > > >>>> > > >>> > > >> > > > https://calcite.apache.org/javadocAggregate/org/apache/calcite/linq4j/RawQueryable.html > > >>> ). > > >>>> It has a method getProvider method that returns QueryProvider. > What > > >>> roles > > >>>> does query provider play ?5. What role does ExtendedQuerable play ? > > >>>> Please let me know if there are any basic > > >>>> examples/blogs/documentation/etc. > > >>>> Thanks for your help. > > >>>> Mans > > >>> > > >> > > > > >
