Re: Using Spark 2.4.0 as Execution Engine with Apache Calcite

2019-08-29 Thread Shubham Kumar
Hey Julian, Thanks, I have subscribed to the list now. On Wed, Aug 28, 2019 at 10:58 PM Julian Hyde wrote: > Your message was received. I guess no one had the time to respond. > > Can you please subscribe to the list? I had to moderate your message > through because you are not a subscriber. >

Re: Using Spark 2.4.0 as Execution Engine with Apache Calcite

2019-08-28 Thread Feng Zhu
Hi,Sgybham, According to our study, it is developed for Spark 1.x and not powerful in production. Rather than relying on Spark adapter, I suggest you convert RelNode back to SQL based on SparkSqlDialect, and then execute them by Spark SQL. Shubham Kumar 于2019年8月29日周四 上午1:26写道: > Hey, > > Does

Re: Using Spark 2.4.0 as Execution Engine with Apache Calcite

2019-08-28 Thread Julian Hyde
Your message was received. I guess no one had the time to respond. Can you please subscribe to the list? I had to moderate your message through because you are not a subscriber. Julian > On Aug 28, 2019, at 9:03 AM, Shubham Kumar wrote: > > Hey, > > Does anyone have any insights on this?

Re: Using Spark 2.4.0 as Execution Engine with Apache Calcite

2019-08-28 Thread Shubham Kumar
Hey, Does anyone have any insights on this? Also wanted to confirm that it is being sent on the mailing list of Calcite. Thanks Shubham On Mon, Aug 26, 2019 at 6:16 PM Shubham Kumar wrote: > Hey, > > I am trying to optimize queries by creating materialized views in Hive and > then exposing

Using Spark 2.4.0 as Execution Engine with Apache Calcite

2019-08-26 Thread Shubham Kumar
Hey, I am trying to optimize queries by creating materialized views in Hive and then exposing them to Calcite for optimized query rewrite. I wanted to know that how mature is the spark adapter of Calcite i.e. Let's say while making the connection I set parameter spark=true