import org.apache.spark.sql.expressions.Window
val partitionBy = Window.partitionBy("name", "sit").orderBy("data_date")
val newDf = df.withColumn("PreviousDate", lag("uniq_im",
1).over(partitionBy))
Cheers...
On Thu, Mar 14, 2019 at 4:55 AM anbu wrote:
> Hi,
>
> To calculate LAG functions difference for the two data_date(current date
> and
> previous date) on the same column
> Could you please help me to implement the below scenario using scala spark
> Dataset.
> uniq_im - LAG(uniq_im,1,0) OVER PARTITION BY(name,sit,plc,country,state)
> order by (data_date) as calc_value.
>
> could you please help me how to implement using spark scala Dataset.
>
>
>
> --
> Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
>
> -
> To unsubscribe e-mail: user-unsubscr...@spark.apache.org
>
>
