Re: On app upgrade, restore sliding window data.

2015-03-03 Thread Matus Faro
Thank you Arush, I've implemented initial data for a windowed operation and opened a pull request here: https://github.com/apache/spark/pull/4875 On Tue, Feb 24, 2015 at 4:49 AM, Arush Kharbanda ar...@sigmoidanalytics.com wrote: I think this could be of some help to you.

Re: On app upgrade, restore sliding window data.

2015-02-24 Thread Arush Kharbanda
I think this could be of some help to you. https://issues.apache.org/jira/browse/SPARK-3660 On Tue, Feb 24, 2015 at 2:18 AM, Matus Faro matus.f...@kik.com wrote: Hi, Our application is being designed to operate at all times on a large sliding window (day+) of data. The operations

On app upgrade, restore sliding window data.

2015-02-23 Thread Matus Faro
Hi, Our application is being designed to operate at all times on a large sliding window (day+) of data. The operations performed on the window of data will change fairly frequently and I need a way to save and restore the sliding window after an app upgrade without having to wait the duration of