Hi Konstantin, What you could do, is that you write and intermediate job that has the old ValueState “oldState” and the new one “newState”, with the new format.
When an element comes in this intermediate job, you check the oldState if it is empty for that key or not. If it is null (empty), you simply process the element as it is the first time you see the key. If it is not empty, then you implement your migration logic that ports the oldState to the newState format, you store the migrated state in the newState, and delete it from the oldState. Of course after the migration you process the element as usual, but only use the new state. If at some point you are sure that you have seen all the keys from the previous version of the code, then at that point you can be sure that all the old-format states have been migrated and you can take a savepoint, clean up the job from the migration logic, and resume from the savepoint with the new code. If there is no such point where you can be sure that you have migrated the state for all keys, then you just your job run like this, i.e. with the migration logic. The problem with the above strategy is that in the case that you do not have a point where you can be sure that you have seen all keys, if you want to migrate once again in the future, you will have to implement the same thing but migrating from two different previous versions. But at that point you may have a policy that says that if I have not seen a key for the last week or month, then I do not consider active and i do not care about it. I hope this helps! Cheers, Kostas > On Mar 14, 2018, at 10:03 AM, kla <lalafar...@gmail.com> wrote: > > Hi Aljoscha, > > Thanks for your reply. > > Do you have a suggestion how can we workaround it ? > > We have a production system running with Flink and it is mandatory to add > one more field in the state. > > Maybe some how we can write our own serializer? > > Thanks, > Konstantin > > > > -- > Sent from: > http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
