Modify savepoints in Flink

2022-10-21 Thread Sriram Ganesh
Hi All, I am working on a scenario where I need to modify the existing savepoint operator state. Ex: Wanted to remove some offset of the savepoint. What is the better practice for these scenarios?. Could you please help me with any example as such? Thanks in advance. -- *Sriram G* *Tech*

Re: Modify savepoints in Flink

2022-10-21 Thread Piotr Nowojski
Hi Sriram, You can read and modify savepoints using StateProcessor API [1]. Alternatively, you can modify a code of your function/operator for which you want to modify the state. For example in the `org.apache.flink.streaming.api.checkpoint.CheckpointedFunction#initializeState` method you could a

Re: Modify savepoints in Flink

2022-10-21 Thread Sriram Ganesh
Thanks !. Will try this. On Fri, Oct 21, 2022 at 2:22 PM Piotr Nowojski wrote: > Hi Sriram, > > You can read and modify savepoints using StateProcessor API [1]. > > Alternatively, you can modify a code of your function/operator for which > you want to modify the state. For example in the > `org.

Re: Modify savepoints in Flink

2022-10-21 Thread Piotr Nowojski
ops > Alternatively, you can modify a code of your function/operator for which you want to modify the state. For example in the `org.apache.flink.streaming.api.checkpoint.CheckpointedFunction#initializeState` method you could add some code that would do a migration of your old state to a new one.

Re: Modify savepoints in Flink

2022-10-21 Thread Sriram Ganesh
I have question on this. Different connector can have different serialisation and de-serlisation technique right?. Wouldn't that impact?. If I use StateProcessor API, would that be agnostic to all the sources and sinks?. On Fri, Oct 21, 2022, 18:00 Piotr Nowojski wrote: > ops > > > Alternatively

Re: Modify savepoints in Flink

2022-10-21 Thread Piotr Nowojski
Hi, Yes and no. StateProcessor API can read any Flink state, but you have to describe the state you want it to access. Take a look at the example in the docs [1]. First you have an example of a theoretical production function `StatefulFunctionWithTime`, which state you want to modify. Note the `V

Re: Modify savepoints in Flink

2022-10-21 Thread Sriram Ganesh
Thanks, I'll check it out. On Fri, Oct 21, 2022, 18:20 Piotr Nowojski wrote: > Hi, > > Yes and no. StateProcessor API can read any Flink state, but you have to > describe the state you want it to access. Take a look at the example in the > docs [1]. > > First you have an example of a theoretical