[
https://issues.apache.org/jira/browse/FLINK-6482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16015315#comment-16015315
]
ASF GitHub Bot commented on FLINK-6482:
---------------------------------------
GitHub user tzulitai opened a pull request:
https://github.com/apache/flink/pull/3937
[FLINK-6482] [core] Add nested serializers to config snapshots of composite
serializers
This commit adds also the nested serializers themselves to the
configuration snapshots of composite serializers. This opens up the opportunity
to use the previous nested serializer as the convert deserializer in the case
that a nested serializer in the new serializer determines that state migration
is required.
Take for example a Tuple serializer. If any one of the nested field
serializers is not compatible, then the whole Tuple serializer will need to be
migrated. Prior to this PR, this means that all serializers would need to be
able to be either compatible or provide a convert deserializer. With this
change, this relaxes this by opening up the opportunity to just use the
previous serializer of fields as the convert deserializer.
Only the composite serializers are affected by this change.
This commit also consolidates all TypeSerializer-related serialization
proxies into a single utility class. A few renamings are included, namely:
1. `ClassNotFoundDummyTypeSerializer` --> `UnloadableDummyTypeSerializer`
2. `TypeSerializerUtil` --> `TypeSerializerSerializationUtil` (because
that's all its used for)
3. `TypeSerializerSerializationProxy` --> functionality moved to a static
method of `TypeSerializerSerializationUtil`
4. `StateMigrationUtil` --> moved to flink-core and renamed to
`CompatibilityUtil`.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/tzulitai/flink FLINK-6482
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/3937.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3937
----
commit a189999f799fe943f2297293a2f74707b9c4188d
Author: Tzu-Li (Gordon) Tai <tzuli...@apache.org>
Date: 2017-05-18T06:51:33Z
[FLINK-6482] [core] Add nested serializers to config snapshots of composite
serializers
This commit adds also the nested serializers themselves to the
configuration snapshots of composite serializers. This opens up the
oppurtunity to use the previous nested serializer as the convert
deserializer in the case that a nested serializer in the new serializer
determines that state migration is required.
This commit also consolidate all TypeSerializer-related serialization
proxies into a single utility class.
----
> Add nested serializers into configuration snapshots of composite serializers
> ----------------------------------------------------------------------------
>
> Key: FLINK-6482
> URL: https://issues.apache.org/jira/browse/FLINK-6482
> Project: Flink
> Issue Type: Improvement
> Components: State Backends, Checkpointing, Type Serialization System
> Reporter: Tzu-Li (Gordon) Tai
> Assignee: Tzu-Li (Gordon) Tai
> Priority: Critical
> Fix For: 1.3.0
>
>
> Currently, the composite serializers' configuration snapshots only wrap the
> config snapshots of nested serializers.
> We should also consider adding serialization of the nested serializers into
> the config snapshot, so that in the case where only some nested serializer
> cannot be loaded (class missing / implementation changed), we can also
> provide a path for serializer upgrades.
> This applies for all composite serializers that have nested serializers.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
