Re: If I pass raw SQL string to dataframe do I still get the Spark SQL optimizations?

2017-07-06 Thread ayan guha
; > On Thu, Jul 6, 2017 at 5:28 PM, kant kodali <kanth...@gmail.com> wrote: > >> HI All, >> >> I am wondering If I pass a raw SQL string to dataframe do I still get the >> Spark SQL optimizations? why or why not? >> >> Thanks! >> > > -- Best Regards, Ayan Guha

Re: If I pass raw SQL string to dataframe do I still get the Spark SQL optimizations?

2017-07-06 Thread Michael Armbrust
It goes through the same optimization pipeline. More in this video <https://youtu.be/1a4pgYzeFwE?t=608>. On Thu, Jul 6, 2017 at 5:28 PM, kant kodali <kanth...@gmail.com> wrote: > HI All, > > I am wondering If I pass a raw SQL string to dataframe do I still get the > Spar

If I pass raw SQL string to dataframe do I still get the Spark SQL optimizations?

2017-07-06 Thread kant kodali
HI All, I am wondering If I pass a raw SQL string to dataframe do I still get the Spark SQL optimizations? why or why not? Thanks!