[jira] [Comment Edited] (FLINK-10897) Support POJO state schema evolution
[
https://issues.apache.org/jira/browse/FLINK-10897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16798569#comment-16798569
]
william hesch edited comment on FLINK-10897 at 3/22/19 6:36 PM:
Does this also cover scala case classes?
Edit - It looks like no. They use the default kryoserializer. Any timeline for
scala case class support?
was (Author: whesch):
Does this also cover scala case classes?
> Support POJO state schema evolution
> ---
>
> Key: FLINK-10897
> URL: https://issues.apache.org/jira/browse/FLINK-10897
> Project: Flink
> Issue Type: Sub-task
> Components: API / Type Serialization System
>Reporter: Tzu-Li (Gordon) Tai
>Assignee: Tzu-Li (Gordon) Tai
>Priority: Major
> Fix For: 1.8.0
>
>
> Main action point for this is to implement a separate POJO serializer that is
> specifically used as the restore serializer.
> This restore POJO serializer should be able to read and dump values of fields
> that no longer exists in the updated POJO schema, and assign default values
> to newly added fields. Snapshot of the {{PojoSerializer}} should contain
> sufficient information so that on restore, the information can be compared
> with the adapted POJO class to figure out which fields have been removed /
> added.
> Changing fields types is out of scope and should not be supported.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (FLINK-10897) Support POJO state schema evolution
[
https://issues.apache.org/jira/browse/FLINK-10897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16798569#comment-16798569
]
william hesch edited comment on FLINK-10897 at 3/22/19 12:06 AM:
-
Does this also cover scala case classes?
was (Author: whesch):
Does this also cover scala case classes?
* [|https://issues.apache.org/jira/secure/AddComment!default.jspa?id=13213003]
> Support POJO state schema evolution
> ---
>
> Key: FLINK-10897
> URL: https://issues.apache.org/jira/browse/FLINK-10897
> Project: Flink
> Issue Type: Sub-task
> Components: API / Type Serialization System
>Reporter: Tzu-Li (Gordon) Tai
>Assignee: Tzu-Li (Gordon) Tai
>Priority: Major
> Fix For: 1.8.0
>
>
> Main action point for this is to implement a separate POJO serializer that is
> specifically used as the restore serializer.
> This restore POJO serializer should be able to read and dump values of fields
> that no longer exists in the updated POJO schema, and assign default values
> to newly added fields. Snapshot of the {{PojoSerializer}} should contain
> sufficient information so that on restore, the information can be compared
> with the adapted POJO class to figure out which fields have been removed /
> added.
> Changing fields types is out of scope and should not be supported.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (FLINK-10897) Support POJO state schema evolution
[
https://issues.apache.org/jira/browse/FLINK-10897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16747409#comment-16747409
]
boshu Zheng edited comment on FLINK-10897 at 1/30/19 2:13 AM:
--
I will try to follow the example that you presented at FlinkForwardChina.
Additionally, I propose to add two fields to `PojoSerializer`, namely
`previousFieldNames` and `previousFieldSerializers` which are snapshotted and
restored by a new `PojoSerializerSnapshot`. When restoring states via
`PojoSerializer#deserialize`, we should be able to figure out which fields have
been removed/added or have a new type with these additional fields.
What do you think? [~tzulitai]
was (Author: kisimple):
I will try to follow the example that you presented at FlinkForwardChain.
Additionally, I propose to add two fields to `PojoSerializer`, namely
`previousFieldNames` and `previousFieldSerializers` which are snapshotted and
restored by a new `PojoSerializerSnapshot`. When restoring states via
`PojoSerializer#deserialize`, we should be able to figure out which fields have
been removed/added or have a new type with these additional fields.
What do you think? [~tzulitai]
> Support POJO state schema evolution
> ---
>
> Key: FLINK-10897
> URL: https://issues.apache.org/jira/browse/FLINK-10897
> Project: Flink
> Issue Type: Sub-task
> Components: Type Serialization System
>Reporter: Tzu-Li (Gordon) Tai
>Assignee: boshu Zheng
>Priority: Major
>
> Main action point for this is to implement a separate POJO serializer that is
> specifically used as the restore serializer.
> This restore POJO serializer should be able to read and dump values of fields
> that no longer exists in the updated POJO schema, and assign default values
> to newly added fields. Snapshot of the {{PojoSerializer}} should contain
> sufficient information so that on restore, the information can be compared
> with the adapted POJO class to figure out which fields have been removed /
> added.
> Changing fields types is out of scope and should not be supported.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
