Hi Blake, I didn’t see any mails on the user mailing list regarding this. Is it still relevant.
From the looks of it you might be interested in the savepoints feature: https://ci.apache.org/projects/flink/flink-docs-release-1.3/setup/savepoints.html <https://ci.apache.org/projects/flink/flink-docs-release-1.3/setup/savepoints.html>. This uses the checkpointing mechanism to savepoint the state of a running job so that the job can be canceled and later restored. Best, Aljoscha > On 21. Apr 2017, at 20:42, Blake Essing <essi...@swbell.net> wrote: > > Sorry all for my earlier messages. I just realized that I'm on the wrong > mailing list for what I'm asking. I will move my question to the user list. > > On Friday, April 21, 2017 11:54 AM, Blake Essing <essi...@swbell.net> > wrote: > > > Sorry for the earlier incomplete email, E-mail editor issue. > I have modified the WordCounter example program to use a NiFi DataStream as > input. I would like to add checkpointing to the program to learn how it > works. I have googled and added many of the checkpointing configuration > values to the StreamExecutionEnvironment but I'm missing something critical > to getting this to work or I'm misinterpreting how the checkpointing should > work. My goal is to save state so that if the program is stopped and then > restarted the counts will continue from the last checkpoint. > The counts variable is populated by > > DataStream<WordCounter> counts = text .flatMap(new Tokenizer()) > .keyBy("word") .reduce(new ReduceFunction<WordCounter>() { public > WordCounter reduce(WordCounter a, WordCounter b) throws Exception { > return new WordCounter(a.getWord, a.getCount() + b.getCount()); } > }) > text is just a DataStream<String> populated from NiFi where lines are broken > up into individual strings. > WordCounter is a simple pojo that stores the word and a count with get/set > methods. > Please let me know if you have any questions about the code. > > On Friday, April 21, 2017 11:48 AM, Blake Essing <essi...@swbell.net> > wrote: > > > I have modified the WordCounter example program to use a NiFi DataStream as > input. I would like to add checkpointing to the program to learn how it > works. I have googled and added many of the checkpointing configuration > values to the StreamExecutionEnvironment but I'm missing something critical > to getting this to work or I'm misinterpreting how the checkpointing should > work. My goal is to save state so that if the program is stopped and then > restarted the counts will continue from the last checkpoint. The counts > variable is populated by > > >
