[
https://issues.apache.org/jira/browse/FLINK-23143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17534149#comment-17534149
]
Hangxiang Yu commented on FLINK-23143:
--------------------------------------
> I like the simplicity of your solution, but I think the concern I expressed
> above (about creation of multiple state objects) is valid.
The added methods we implement just mock what we do when restore (from rocksdb
or hashmap).
They will judge before adding new state. And judging will also works when users
access.
(We don't implement them by createInternalState)
So I think it's not a problem ?
> Besides that, RegisteredKeyValueStateBackendMetaInfo seems implementation
> detail of a state backend, so it shouldn't be exposed.
You are right. Maybe we could introduce a new more universal class to wrap
these information, WDYT?
> Regarding the key serializer upgrade, I think it's [not
> supported|https://nightlies.apache.org/flink/flink-docs-master/docs/dev/datastream/fault-tolerance/serialization/schema_evolution/]
> currently and therefore is out of the scope of this ticket.
I agree. But we haven't implement the logic of compatibility check for key
serializer as you could see in rocksdb and heap.
What I describe above about key serializer is the difficulty of implementing
the logic of compatibility check for key serializer.
It will also works even if we lack it. Do you think it doesn't matter ?
> Support state migration
> -----------------------
>
> Key: FLINK-23143
> URL: https://issues.apache.org/jira/browse/FLINK-23143
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / State Backends
> Reporter: Roman Khachatryan
> Priority: Minor
> Labels: pull-request-available
> Fix For: 1.16.0
>
>
> ChangelogKeyedStateBackend.getOrCreateKeyedState is currently used during
> recovery; on 1st user access, it doesn't update metadata nor migrate state
> (as opposed to other backends).
>
> The proposed solution is to
> # wrap serializers (and maybe other objects) in getOrCreateKeyedState
> # store wrapping objects in a new map keyed by state name
> # pass wrapped objects to delegatedBackend.createInternalState
> # on 1st user access, lookup wrapper and upgrade its wrapped serializer
> This should be done for both KV/PQ states.
>
> See also [https://github.com/apache/flink/pull/15420#discussion_r656934791]
>
> cc: [~yunta]
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
