Hi Alexis, If we change the operator uid and restart the job, the job will not be started successfully[1]. We have to use --allowNonRestoredState to start it. This means that the state for the old uid will not be used in the operator with the new uid. I think the data in the state will be lost.
Best, Hang [1] https://nightlies.apache.org/flink/flink-docs-release-1.16/docs/ops/state/savepoints/#what-happens-if-i-delete-an-operator-that-has-state-from-my-job Alexis Sarda-Espinosa <sarda.espin...@gmail.com> 于2023年2月13日周一 19:56写道： > Hi Hang, > > Thanks for the confirmation. One follow-up question with a somewhat > convoluted scenario: > > 1. An unaligned checkpoint is created. > 2. I stop the job *without* savepoint. > 3. I want to start a modified job from the checkpoint, but I changed > one of the operator's uids. > > If the operator whose uid changed had in-flight data as part of the > checkpoint, it will lose said data after starting, right? > > I imagine this is not good practice, but it's just a hypothetical scenario > I wanted to understand better. > > Regards, > Alexis. > > > Am Mo., 13. Feb. 2023 um 12:33 Uhr schrieb Hang Ruan < > ruanhang1...@gmail.com>: > >> ps: the savepoint will also not contain in-flight data. >> >> Best, >> Hang >> >> Hang Ruan <ruanhang1...@gmail.com> 于2023年2月13日周一 19:31写道： >> >>> Hi Alexis, >>> >>> No, aligned checkpoint will not contain the in-flight. Aligned >>> checkpoint makes sure that the data before the barrier has been processed >>> and there is no need to store in-flight data for one checkpoint. >>> >>> I think these documents[1][2] will help you to understand it. >>> >>> >>> Best, >>> Hang >>> >>> [1] >>> https://nightlies.apache.org/flink/flink-docs-release-1.16/docs/ops/state/checkpointing_under_backpressure/ >>> [2] >>> https://nightlies.apache.org/flink/flink-docs-release-1.16/docs/concepts/stateful-stream-processing/#checkpointing >>> >>> Alexis Sarda-Espinosa <sarda.espin...@gmail.com> 于2023年2月11日周六 06:00写道： >>> >>>> Hello, >>>> >>>> One feature of unaligned checkpoints is that the checkpoint barriers >>>> can overtake in-flight data, so the buffers are persisted as part of the >>>> state. >>>> >>>> The documentation for savepoints doesn't mention anything explicitly, >>>> so just to be sure, will savepoints always wait for in-flight data to be >>>> processed before they are completed, or could they also persist buffers in >>>> certain situations (e.g. when there's backpressure)? >>>> >>>> Regards, >>>> Alexis. >>>> >>>>
