Very nice suggestion. I wonder can we introduce this feature by config? Maybe it’s better for users using more than one query engine to interpret and execute query.
On Fri, Aug 4, 2023 at 22:03 Alessandro Solimando < alessandro.solima...@gmail.com> wrote: > Hello, > as LakeShen suggests, you can take a look into RexUtil#expandSearch, you > can see it in action in RexProgramTest tests, one example: > > > https://github.com/apache/calcite/blob/98f3048fb1407e2878162ffc80388d4f9dd094b2/core/src/test/java/org/apache/calcite/rex/RexProgramTest.java#L1710-L1727 > > Best regards, > Alessandro > > On Fri, 4 Aug 2023 at 15:45, LakeShen <shenleifight...@gmail.com> wrote: > > > Hi P.F.ZHAN,in calcite,it has a method RexUtil#expandSearch to expand > > Search,maybe you could get some information from this method. > > > > There is also some logic to simplify Search in the > > RexSimplify#simplifySearch method. I hope this could help you. > > Here's the code: 1. > > > > > https://github.com/apache/calcite/blob/98f3048fb1407e2878162ffc80388d4f9dd094b2/core/src/main/java/org/apache/calcite/rex/RexUtil.java#L593 > > 2. > > > > > https://github.com/apache/calcite/blob/98f3048fb1407e2878162ffc80388d4f9dd094b2/core/src/main/java/org/apache/calcite/rex/RexSimplify.java#L2132 > > > > Best, > > LakeShen > > > > Soumyadeep Mukhopadhyay <soumyamy...@gmail.com> 于2023年8月4日周五 20:29写道： > > > > > Thank you, shall explore more on this! :) > > > > > > > > > On Fri, 4 Aug 2023 at 5:53 PM, P.F. ZHAN <dethr...@gmail.com> wrote: > > > > > > > Aha, I'm using Apache Kylin which uses Calcite to generate a logical > > > plan, > > > > then convert to Spark plan to execute a query. Given that Calcite has > > > more > > > > operations for aggregations, and Kylin wants to take full advantage > of > > > > precomputed cubes (something like Calcite's materialized views), it > > uses > > > > both Calcite and Spark(for distribution computing). Maybe it's wild > > and a > > > > little fun, but it does works well on many scenarios. > > > > > > > > On Fri, Aug 4, 2023 at 8:10 PM Soumyadeep Mukhopadhyay < > > > > soumyamy...@gmail.com> wrote: > > > > > > > > > I am curious about your use case. Are you not losing out on the > > > > > optimisations of Calcite when you are using Spark? Is it possible > for > > > you > > > > > to share a general approach where we will be able to keep the > > > > optimisations > > > > > done by Calcite and use Spark on top of it? > > > > > > > > > > > > > > > On Fri, 4 Aug 2023 at 5:19 PM, P.F. ZHAN <dethr...@gmail.com> > wrote: > > > > > > > > > > > Generally speaking, the SEARCH operator is very good, but when we > > use > > > > > > Calcite to optimize the logical plan and then use Spark to > execute, > > > > this > > > > > is > > > > > > unsupported. So is there a more elegant way to close the SEARCH > > > > operator? > > > > > > Or how to convert the SEARCH operator to the IN operator before > > > > > converting > > > > > > the Calcite logical plan to the Spark logical plan? If we do > this, > > we > > > > > need > > > > > > to consider Join / Filter, are there any other RelNodes? > > > > > > > > > > > > Maybe, this optimization is optional more better at present for > > many > > > > > query > > > > > > execution engine does not support this operator? > > > > > > > > > > > > > > > > > > > > >
