Aljoscha Krettek created FLINK-6383:
---------------------------------------
Summary: Add versioning for AbstractStreamOperator state
Key: FLINK-6383
URL: https://issues.apache.org/jira/browse/FLINK-6383
Project: Flink
Issue Type: Improvement
Components: DataStream API
Affects Versions: 1.3.0
Reporter: Aljoscha Krettek
Assignee: Aljoscha Krettek
Priority: Blocker
Fix For: 1.3.0
Right now, {{AbstractStreamOperator}} checkpoints key-group state in
{{AbstractStreamOperator.snapshotState(StateSnapshotContext context)}}.
Currently, this is the timers for each key group. We might want to change the
format of this or what we store in the future. In order to facilitate this we
should write a version to the beginning of the stream so that we can deal with
schema updates.
For dealing with the fact that we don't currently write a version number we
have to at a magic number add the beginning (think {{0xCAFEBABE}}). For dealing
with state that was written prior to versioning we read the first 8 bytes,
buffer them, if they match the magic number we read the version and from then
on jump to the logic based on the version. If the magic number doesn't match we
know we have state from a pre-version version and read the stream using the old
code, including the 8 buffered bytes.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
