Re: Iterate over grouped df to create new rows/df

2017-07-10 Thread ayan guha
Hi Happy that my solution worked for you. The solution is a sql trick to identify the boundaries of a session. It has nothing to do with spark itself. In the first step it calculates the difference between two consecutive rows. Then it gives a number fg which is a running number, remains same

Iterate over grouped df to create new rows/df

2017-07-07 Thread Junaid Nasir
Hi everyone, I am kind of stuck in a problem and was hoping for some pointers or help :) have tried different things but couldn't achieve the desired results. I want to *create single row from multiple rows if those rows are continuous* (based on time i.e if next row's time is within 2 minutes