Re: Does state survive application restart in StatefulNetworkWordCount?

2016-01-04 Thread Tathagata Das
It does get recovered if you restart from checkpoints. See the example RecoverableNetworkWordCount.scala On Sat, Jan 2, 2016 at 6:22 AM, Rado Buranský wrote: > I am trying to understand how state in Spark Streaming works in general. > If I run this example program twice

Re: Does state survive application restart in StatefulNetworkWordCount?

2016-01-04 Thread Rado Buranský
I asked the question on Twitter and got this response: https://twitter.com/jaceklaskowski/status/683923649632579588 Is Jacek right? If you stop and then start the application correctly then the state is not recovered? It is recovered only in case of failure? On Mon, Jan 4, 2016 at 8:19 PM,

Does state survive application restart in StatefulNetworkWordCount?

2016-01-02 Thread Rado Buranský
I am trying to understand how state in Spark Streaming works in general. If I run this example program twice will the second run see state from the first run? https://github.com/apache/spark/blob/master/examples/src/main/scala/org/apache/spark/examples/streaming/StatefulNetworkWordCount.scala It