Re: Does state survive application restart in StatefulNetworkWordCount?

[Tathagata Das]

It does get recovered if you restart from checkpoints. See the example
RecoverableNetworkWordCount.scala

On Sat, Jan 2, 2016 at 6:22 AM, Rado Buranský 
wrote:

> I am trying to understand how state in Spark Streaming works in general.
> If I run this example program twice will the second run see state from the
> first run?
>
> https://github.com/apache/spark/blob/master/examples/src/main/scala/org/apache/spark/examples/streaming/StatefulNetworkWordCount.scala
>
> It seems no. Is there a way how to achieve this? I am thinking about
> redeploying an application an I would like not to loose the current state.
>
> Thanks
>

Re: Does state survive application restart in StatefulNetworkWordCount?

[Rado Buranský]

I asked the question on Twitter and got this response:
https://twitter.com/jaceklaskowski/status/683923649632579588

Is Jacek right? If you stop and then start the application correctly then
the state is not recovered? It is recovered only in case of failure?

On Mon, Jan 4, 2016 at 8:19 PM, Tathagata Das  wrote:

> It does get recovered if you restart from checkpoints. See the example
> RecoverableNetworkWordCount.scala
>
> On Sat, Jan 2, 2016 at 6:22 AM, Rado Buranský 
> wrote:
>
>> I am trying to understand how state in Spark Streaming works in general.
>> If I run this example program twice will the second run see state from the
>> first run?
>>
>> https://github.com/apache/spark/blob/master/examples/src/main/scala/org/apache/spark/examples/streaming/StatefulNetworkWordCount.scala
>>
>> It seems no. Is there a way how to achieve this? I am thinking about
>> redeploying an application an I would like not to loose the current state.
>>
>> Thanks
>>
>
>

Does state survive application restart in StatefulNetworkWordCount?

[Rado Buranský]

I am trying to understand how state in Spark Streaming works in general. If
I run this example program twice will the second run see state from the
first run?
https://github.com/apache/spark/blob/master/examples/src/main/scala/org/apache/spark/examples/streaming/StatefulNetworkWordCount.scala

It seems no. Is there a way how to achieve this? I am thinking about
redeploying an application an I would like not to loose the current state.

Thanks