Tzu-Li (Gordon) Tai created FLINK-11947:
-------------------------------------------
Summary: Support MapState key / value schema evolution for RocksDB
Key: FLINK-11947
URL: https://issues.apache.org/jira/browse/FLINK-11947
Project: Flink
Issue Type: Improvement
Components: API / Type Serialization System, Runtime / State Backends
Reporter: Tzu-Li (Gordon) Tai
Currently, we do not attempt to perform state schema evolution if the key or
value's schema of a user {{MapState}} has changed when using {{RocksDB}}:
https://github.com/apache/flink/blob/953a5ffcbdae4115f7d525f310723cf8770779df/flink-state-backends/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java#L542
This was disallowed in the initial support for state schema evolution because
the way we did state evolution in the RocksDB state backend was simply
overwriting values.
For {{MapState}} key evolution, only overwriting RocksDB values does not work,
since RocksDB entries for {{MapState}} uses a composite key containing the map
state key. This means that when evolving {{MapState}} in this case with an
evolved key schema, we will have new entries.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
