Using checkpointing for graceful updates is my understanding as well, based on the writeup in https://spark.apache.org/docs/latest/structured-streaming-programming-guide.html#recovering-from-failures-with-checkpointing, and some prototyping. Have you faced any missed events?
On Mon, Dec 17, 2018 at 6:56 PM Yuta Morisawa <yu-moris...@kddi-research.jp> wrote: > Hi > > Now I'm trying to update my structured streaming application. > But I have no idea how to update it gracefully. > > Should I stop it, replace a jar file then restart it? > In my understanding, in that case, all the state will be recovered if I > use checkpoints. > Is this correct? > > Thank you, > > > -- > > > --------------------------------------------------------------------- > To unsubscribe e-mail: user-unsubscr...@spark.apache.org > >
