Re: Execution plan in spark

There is no execution plan for FP. Execution plan exists for sql. Regards Sab On 24-Feb-2016 2:46 pm, "Ashok Kumar" wrote: > Gurus, > > Is there anything like explain in Spark to see the execution plan in > functional programming? > > warm regards >

Re: Execution plan in spark

Also bear in mind that explain() method call works on transformations (Transformations are just manipulations of the data.). examples filter, map, orderBy etc scala> var y = HiveContext.table("sales").select("time_id").agg(max("time_id")).explain(true) == Parsed Logical Plan ==

Re: Execution plan in spark

looks useful thanks On Wednesday, 24 February 2016, 9:42, Yin Yang wrote: Is the following what you were looking for ?     sqlContext.sql("""    CREATE TEMPORARY TABLE partitionedParquet    USING org.apache.spark.sql.parquet    OPTIONS (      path '/tmp/partitioned' 

Re: Execution plan in spark

Is the following what you were looking for ? sqlContext.sql(""" CREATE TEMPORARY TABLE partitionedParquet USING org.apache.spark.sql.parquet OPTIONS ( path '/tmp/partitioned' )""") table("partitionedParquet").explain(true) On Wed, Feb 24, 2016 at 1:16 AM, Ashok

Execution plan in spark

Gurus, Is there anything like explain in Spark to see the execution plan in functional programming? warm regards