Re: spark 1.5 sort slow

Can you include the output of `explain()` for each of the runs? On Tue, Sep 1, 2015 at 1:06 AM, patcharee wrote: > Hi, > > I found spark 1.5 sorting is very slow compared to spark 1.4. Below is my > code snippet > > val sqlRDD = sql("select date, u, v, z from

spark 1.5 sort slow

Hi, I found spark 1.5 sorting is very slow compared to spark 1.4. Below is my code snippet val sqlRDD = sql("select date, u, v, z from fino3_hr3 where zone == 2 and z >= 2 and z <= order by date, z") println("sqlRDD " + sqlRDD.count()) The fino3_hr3 (in the sql command) is a hive