Hi Gordon, I noticed there has been no movement on this issue and I'm wondering if I can find some way to work around this. My MapState value is a case class container of Avro-generated SpecificRecords. If one SpecificRecord changes I am stuck.
>From the issue It seems like the blocker is around evolving the MapState key type. That may be a nasty problem, but my key type is stable and will never change. What do you think the level of difficulty would be to add support for evolving only the value? Also, if I use GenericRecord instead of SpecificRecord will the need for schema evolution still be triggered? Or does it actually go down to the avro schema rather than just the class serialVersionUID? On Mon, Mar 18, 2019 at 1:10 AM Tzu-Li (Gordon) Tai <tzuli...@apache.org> wrote: > Hi Cliff, > > Thanks for bringing this up! > AFAIK, this wouldn't be a long-term blocker. I've just opened a JIRA to > track this [1]. > > As explained in the JIRA ticket, the main reason this is disallowed in the > initial support for state schema evolution was due to how migration was > implemented in the RocksDB state backend. > Technically speaking, enabling this in the future is definitely possible. > > Cheers, > Gordon > > [1] https://issues.apache.org/jira/browse/FLINK-11947 > > On Mon, Mar 18, 2019 at 11:20 AM Cliff Resnick <cre...@gmail.com> wrote: > >> After trying out state migration in 1.8 rc2 I ran into this hard stop >> below. The comment does not give an indication why rocksdb map state cannot >> be migrated, and I'm wondering what the status is, since we do need this >> functionality and would like to know if this is a long-term blocker or not. >> Anyone know? >> >> >> https://github.com/apache/flink/blob/953a5ffcbdae4115f7d525f310723cf8770779df/flink-state-backends/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java#L542 >> >
