Re: How to deal with context dependent computing?

Hi Junfeng, You should be able to do this with window aggregation functions lead or lag https://jaceklaskowski.gitbooks.io/mastering-spark-sql/spark-sql-functions.html#lead Thanks, Dev On Mon, Aug 27, 2018 at 7:08 AM JF Chen wrote: > Thanks Sonal. > For example, I have data as following: >

Re: How to deal with context dependent computing?

Thanks Sonal. For example, I have data as following: login 2018/8/27 10:00 logout 2018/8/27 10:05 login 2018/8/27 10:08 logout 2018/8/27 10:15 login 2018/8/27 11:08 logout 2018/8/27 11:32 Now I want to calculate the time between each login and logout. For example, I should get 5 min, 7 min, 24

Re: How to deal with context dependent computing?

Hi Junfeng, Can you please show by means of an example what you are trying to achieve? Thanks, Sonal Nube Technologies On Thu, Aug 23, 2018 at 8:22 AM, JF Chen wrote: > For example, I have some data with timstamp marked as

How to deal with context dependent computing?

For example, I have some data with timstamp marked as category A and B, and ordered by time. Now I want to calculate each duration from A to B. In normal program, I can use the flag bit to record the preview data if it is A or B, and then calculate the duration. But in Spark Dataframe, how to do