Re: Spark SQL lag() window function, strange behavior

Hi Ross, What version of spark are you using? There were two issues that affected the results of window function in Spark 1.5 branch. Both of issues have been fixed and will be released with Spark 1.5.2 (this release will happen soon). For more details of these two issues, you can take a look at

Spark SQL lag() window function, strange behavior

Hello Spark community - I am running a Spark SQL query to calculate the difference in time between consecutive events, using lag(event_time) over window - SELECT device_id, unix_time, event_id, unix_time - lag(unix_time) OVER (PARTITION BY device_id ORDER

Re: Spark SQL lag() window function, strange behavior

I am using Spark 1.5.0 on Yarn On Nov 2, 2015, at 3:16 PM, Yin Huai > wrote: Hi Ross, What version of spark are you using? There were two issues that affected the results of window function in Spark 1.5 branch. Both of issues have been fixed