Re: Spark streaming not remembering previous state

2016-02-27 Thread Vinti Maheshwari
urday, February 27, 2016 at 3:28 PM > To: user <user@spark.apache.org> > Subject: Spark streaming not remembering previous state > > Hi All, > > I wrote spark streaming program with stateful transformation. > It seems like my spark streaming application is doing computation

Re: Spark streaming not remembering previous state

2016-02-27 Thread Sebastian Piu
Here: https://github.com/apache/spark/blob/master/examples/src/main/scala/org/apache/spark/examples/streaming/RecoverableNetworkWordCount.scala On Sat, 27 Feb 2016, 20:42 Sebastian Piu, wrote: > You need to create the streaming context using an existing checkpoint for >

Re: Spark streaming not remembering previous state

2016-02-27 Thread Sebastian Piu
You need to create the streaming context using an existing checkpoint for it to work See sample here On Sat, 27 Feb 2016, 20:28 Vinti Maheshwari, wrote: > Hi All, > > I wrote spark streaming program with stateful transformation. > It seems like my spark streaming