Objects with fields that are not serializable

[Dominik Wosiński]

Hey,
I have a question about using classes with fields that are not serializable
in DataStream. Basically, I would like to use the  Java's Optional in
DataStream. So Say I have a class *Data *that has several optional fields
and I would like to have *DataStream*. I don't think this should
cause any issues, but I thought it can be good to ask whether this can
cause any issues with Flink Jobs.

Thanks,
Best,
Dom.

Re: Objects with fields that are not serializable

[Timo Walther]

Hi Dominik,

Flink does not use Java serialization logic for network communication. 
So objects must not implement `Serializable` for usage during runtime 
(DataStream).


Only if those classes are member variables of a Function like 
MapFunction, they need to serializable to ship the function code to the 
cluster.


You can check which serializer and nested serializers are used by calling:

dataStream.getType().createTypeSerializer(env.getExecutionConfig)

Regards,
Timo



On 14.04.20 12:55, Dominik Wosiński wrote:

Hey,
I have a question about using classes with fields that are not 
serializable in DataStream. Basically, I would like to use the  Java's 
Optional in DataStream. So Say I have a class /Data /that has several 
optional fields and I would like to have /DataStream/. I don't 
think this should cause any issues, but I thought it can be good to ask 
whether this can cause any issues with Flink Jobs.


Thanks,
Best,
Dom.