[
https://issues.apache.org/jira/browse/FLINK-19176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17195551#comment-17195551
]
Igal Shilman commented on FLINK-19176:
--------------------------------------
Hi [~galenwarren],
Thanks for working on this.
I'd be happy to validate that approach with you and see a PR comes out of it!
The rough sketch for the approach that we had in mind was:
1. Add a MessageFactoryKey org.apache.flink.statefun.flink.core.message
package that would contain the information needed to identify the necessary
MessagePayloadSerializer. I think that a simple string field would be enough,
it would contain either one of the predfeined enum values at
MessageFactoryType, or a class name to be loaded dynamically.
2. Add a static factory method to MessageFactory, to accept a
MessageFactoryKey. and it will create message factory with the correct payload
serializer (either one of the pre-defined ones or an instance of class as
specified in the key)
3. Replace every usage of MessageFactoryType with MessageFactoryKey
4. in StatefulFunctionConfig, class make sure that what is returned is
MessageFactoryKey instead of MessageFactoryType.
This should do it!
Good luck,
Igal.
> Support ScalaPB as a message payload serializer in Stateful Functions
> ---------------------------------------------------------------------
>
> Key: FLINK-19176
> URL: https://issues.apache.org/jira/browse/FLINK-19176
> Project: Flink
> Issue Type: Improvement
> Components: Stateful Functions
> Affects Versions: 2.0.0
> Reporter: Galen Warren
> Priority: Major
> Fix For: statefun-2.1.0
>
>
> Currently, Stateful Functions supports four options for serialization of
> message payloads:
> * Protobuf (based on code generated for Java)
> * Kryo
> * Multilanguage
> * Raw
> This proposal is to add a fifth option to this list, to support serialization
> of message payloads based on [ScalaPB |[https://scalapb.github.io/docs/]].
> This would allow Scala developers to use ScalaPB-generated classes as message
> types in Stateful Functions directly as message types, without having to
> convert to/from code generated for Java and/or raw byte[] messages.
> This would be a simple implementation, as there is already a
> [MessagePayloadSerializer
> |[https://github.com/apache/flink-statefun/blob/8ffe619a94917d676cf1054c8a0e60de544663db/statefun-flink/statefun-flink-core/src/main/java/org/apache/flink/statefun/flink/core/message/MessagePayloadSerializer.java]]
> interface that is implemented for each of the existing serialization
> methods; supporting ScalaPB would require a new class implementing
> MessagePayloadSerializer in terms of ScalaPB-generated code, and a means to
> select it, by adding a new value to the [MessageFactoryType
> |[https://github.com/apache/flink-statefun/blob/8ffe619a94917d676cf1054c8a0e60de544663db/statefun-flink/statefun-flink-core/src/main/java/org/apache/flink/statefun/flink/core/message/MessageFactoryType.java]]enumeration.
> Plus testing :)
> I've done this already locally, the implementation is very similar to the
> existing [MessagePayloadSerializerPb
> |[https://github.com/apache/flink-statefun/blob/8ffe619a94917d676cf1054c8a0e60de544663db/statefun-flink/statefun-flink-core/src/main/java/org/apache/flink/statefun/flink/core/message/MessagePayloadSerializerPb.java]]implementation.
> It uses a reference to ScalaPB in "provided" scope.
> Would you be interested in a pull request to add this? Primary benefit is to
> make it easy to use Stateful Functions in a Scala environment. Thanks.
>
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
