unsubscribe

2019-04-28 Thread mayur bhole

How Spark sql query optimisation work if we are using .rdd action ?

2016-08-13 Thread mayur bhole
HI All, Lets say, we have val df = bigTableA.join(bigTableB,bigTableA("A")===bigTableB("A"),"left") val rddFromDF = df.rdd println(rddFromDF.count) My understanding is that spark will convert all data frame operations before "rddFromDF.count" into RDD equivalent operation as we are not