[
https://issues.apache.org/jira/browse/FLINK-19176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17195596#comment-17195596
]
Igal Shilman commented on FLINK-19176:
--------------------------------------
I've went through all the classes that are marked as Serializable, and all of
them are safe to change.
The Java serialization is used to send some of the instances from the planning
phase to the execution phase, but it is always happing at runtime (nothing is
being Java serialized into a persisted storage across executions)
But you did raised a very good point, and there is an additional step to be
made with the MessageTypeSerializer:
At the bottom of that class you would find a Snapshot class. This is indeed
being persisted as a part of the check-pointed state.
You can bump the version to 2, and make sure to differentiate between version 1
and 2 in readSnapshot, you can also change the constructor to accept a Key
instead of a Type now.
Here is an additional information about TypeSerializers in Flink if you are
interested:
https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/state/custom_serialization.html#state-serializers-and-schema-evolution
> Support ScalaPB as a message payload serializer in Stateful Functions
> ---------------------------------------------------------------------
>
> Key: FLINK-19176
> URL: https://issues.apache.org/jira/browse/FLINK-19176
> Project: Flink
> Issue Type: Improvement
> Components: Stateful Functions
> Affects Versions: 2.0.0
> Reporter: Galen Warren
> Priority: Major
> Fix For: statefun-2.1.0
>
>
> Currently, Stateful Functions supports four options for serialization of
> message payloads:
> * Protobuf (based on code generated for Java)
> * Kryo
> * Multilanguage
> * Raw
> This proposal is to add a fifth option to this list, to support serialization
> of message payloads based on [ScalaPB |[https://scalapb.github.io/docs/]].
> This would allow Scala developers to use ScalaPB-generated classes as message
> types in Stateful Functions directly as message types, without having to
> convert to/from code generated for Java and/or raw byte[] messages.
> This would be a simple implementation, as there is already a
> [MessagePayloadSerializer
> |[https://github.com/apache/flink-statefun/blob/8ffe619a94917d676cf1054c8a0e60de544663db/statefun-flink/statefun-flink-core/src/main/java/org/apache/flink/statefun/flink/core/message/MessagePayloadSerializer.java]]
> interface that is implemented for each of the existing serialization
> methods; supporting ScalaPB would require a new class implementing
> MessagePayloadSerializer in terms of ScalaPB-generated code, and a means to
> select it, by adding a new value to the [MessageFactoryType
> |[https://github.com/apache/flink-statefun/blob/8ffe619a94917d676cf1054c8a0e60de544663db/statefun-flink/statefun-flink-core/src/main/java/org/apache/flink/statefun/flink/core/message/MessageFactoryType.java]]enumeration.
> Plus testing :)
> I've done this already locally, the implementation is very similar to the
> existing [MessagePayloadSerializerPb
> |[https://github.com/apache/flink-statefun/blob/8ffe619a94917d676cf1054c8a0e60de544663db/statefun-flink/statefun-flink-core/src/main/java/org/apache/flink/statefun/flink/core/message/MessagePayloadSerializerPb.java]]implementation.
> It uses a reference to ScalaPB in "provided" scope.
> Would you be interested in a pull request to add this? Primary benefit is to
> make it easy to use Stateful Functions in a Scala environment. Thanks.
>
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
