Hi Yun, Thanks for your response. If I understand correctly, for a particular state descriptor you want to improve the evolution of the underlying data structure? Are there other typical scenario’s that you encounter when running a stateful dataflow graph (e.g. merging data structures, changing underlying class (not modifying)) etc.
Best, Marlo > Op 9 jun. 2021, om 10:10 heeft Yun Tang <myas...@live.com> het volgende > geschreven: > > Hi Marlo, > > One of the scenarios that we're trying to improve is to add or remove one > field in state serializer. > Users might add or remove one field during their schema evolution, state > processor could help it with another offline job while state migration could > help it once we restart the new job. > > Best > Yun Tang > ________________________________ > From: Marlo Ploemen <marlo.ploe...@icloud.com.INVALID> > Sent: Wednesday, June 9, 2021 15:57 > To: dev@flink.apache.org <dev@flink.apache.org> > Subject: State migration scenario's > > Hi community, > > I am looking into the data migration and schema evolution process for > stateful streaming jobs. Currently, there is no orchestration support for > performing these job evolutions and no in-job state migration or schema > evolution syntax (as this is part of the separate state processor API). I am > looking for examples (e.g. Github repositories) or scenarios of stateful > streaming jobs where the orchestration of their state evolution process can > improve development quality. > > Best, Marlo
