reduceByKeyAndWindow with initial state

2015-07-10 Thread Imran Alam
We have a streaming job that makes use of reduceByKeyAndWindow https://github.com/apache/spark/blob/v1.4.0/streaming/src/main/scala/org/apache/spark/streaming/dstream/PairDStreamFunctions.scala#L334-L341. We want this to work with an initial state. The idea is to avoid losing state if the

Re: reduceByKeyAndWindow with initial state

2015-07-10 Thread Tathagata Das
Are you talking about reduceByKeyAndWindow with or without inverse reduce? TD On Fri, Jul 10, 2015 at 2:07 AM, Imran Alam im...@newscred.com wrote: We have a streaming job that makes use of reduceByKeyAndWindow