Changes in streaming query that allow or disallow recovery from checkpoint is clearly provided in https://spark.apache.org/docs/latest/structured-streaming-programming-guide.html#recovery-semantics-after-changes-in-a-streaming-query .
On Tue, Dec 18, 2018 at 9:45 AM vincent gromakowski < vincent.gromakow...@gmail.com> wrote: > Checkpointing is only used for failure recovery not for app upgrades. You > need to manually code the unload/load and save it to a persistent store > > Le mar. 18 déc. 2018 à 17:29, Priya Matpadi <pmatp...@gmail.com> a écrit : > >> Using checkpointing for graceful updates is my understanding as well, >> based on the writeup in >> https://spark.apache.org/docs/latest/structured-streaming-programming-guide.html#recovering-from-failures-with-checkpointing, >> and some prototyping. Have you faced any missed events? >> >> On Mon, Dec 17, 2018 at 6:56 PM Yuta Morisawa < >> yu-moris...@kddi-research.jp> wrote: >> >>> Hi >>> >>> Now I'm trying to update my structured streaming application. >>> But I have no idea how to update it gracefully. >>> >>> Should I stop it, replace a jar file then restart it? >>> In my understanding, in that case, all the state will be recovered if I >>> use checkpoints. >>> Is this correct? >>> >>> Thank you, >>> >>> >>> -- >>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe e-mail: user-unsubscr...@spark.apache.org >>> >>>
