Re: Cumulative Sum function using Dataset API

2016-08-09 Thread Jon Barksdale
Cool, learn something new every day. Thanks again. On Tue, Aug 9, 2016 at 4:08 PM ayan guha <guha.a...@gmail.com> wrote: > Thanks for reporting back. Glad it worked for you. Actually sum with > partitioning behaviour is same in oracle too. > On 10 Aug 2016 03:01, "Jon Ba

Re: Cumulative Sum function using Dataset API

2016-08-09 Thread Jon Barksdale
:) Thank you both for your help, Jon On Tue, Aug 9, 2016 at 3:01 AM Santoshakhilesh <santosh.akhil...@huawei.com> wrote: > You could check following link. > > > http://stackoverflow.com/questions/35154267/how-to-compute-cumulative-sum-using-spark > > > > *From:* Jo

Re: Cumulative Sum function using Dataset API

2016-08-08 Thread Jon Barksdale
I don't think that would work properly, and would probably just give me the sum for each partition. I'll give it a try when I get home just to be certain. To maybe explain the intent better, if I have a column (pre sorted) of (1,2,3,4), then the cumulative sum would return (1,3,6,10). Does that