pySpark window functions are not working in the same way as Spark/Scala ones

2015-09-03 Thread Sergey Shcherbakov
Hello all, I'm experimenting with Spark 1.4.1 window functions and have come to a problem in pySpark that I've described in a Stackoverflow question In essence, the wSpec = Window.orderBy(df.a)

Re: pySpark window functions are not working in the same way as Spark/Scala ones

2015-09-03 Thread Davies Liu
This is an known but in 1.4.1, fixed in 1.4.2 and 1.5 (both are not released yet). On Thu, Sep 3, 2015 at 7:41 AM, Sergey Shcherbakov wrote: > Hello all, > > I'm experimenting with Spark 1.4.1 window functions > and have come to a problem in pySpark that I've