Hi JingsongLee, thank you for your answer. I wanted to explore it as the last chance honestly. Anyway if defining custom serializers and types information involves quite a big effort, I would reconsider my guess.
Cheers, Il giorno gio 4 lug 2019 alle ore 08:46 JingsongLee <lzljs3620...@aliyun.com> ha scritto: > Hi Andrea: > Why not make your *MyClass* POJO? [1] If it is a POJO, then flink > will use PojoTypeInfo and PojoSerializer that have a good > implementation already. > > [1] > https://ci.apache.org/projects/flink/flink-docs-release-1.8/dev/types_serialization.html#rules-for-pojo-types > > Best, JingsongLee > > ------------------------------------------------------------------ > From:Andrea Spina <andrea.sp...@radicalbit.io> > Send Time:2019年7月4日(星期四) 14:37 > To:user <user@flink.apache.org> > Subject:Providing Custom Serializer for Generic Type > > Dear community, > in my job, I run with a custom event type *MyClass* which is a sort of > "generic event" that I handle all along my streaming flow both as an event > (DataStream[MyClass]) and as a managed state. > > I see that Flink warns me about generic serialization of > *MyClass* > INFO [run-main-0] (TypeExtractor.java:1818) - class io.radicalbit.MyClass > does not contain a setter for field io$radicalbit$MyClass$$schema > INFO [run-main-0] (TypeExtractor.java:1857) - Class class > io.radicalbit.MyClass cannot be used as a POJO type because not all fields > are valid POJO fields, and must be processed as GenericType. Please read > the Flink documentation on "Data Types & Serialization" for details of the > effect on performance. > INFO [run-main-0] (TypeExtractor.java:1818) - class io.radicalbit.MyClass > does not contain a setter for field io$radicalbit$MyClass$schema > > So that I wanted to provide my custom serializer for MyClass, trying first > to register the Java one to check if the system recognizes it so I followed > [1] but it seems that it is not considered. > > I read then about [2] (the case is way akin to mine) and AFAIU I need to > implement a custom TypeInformation and TypeSerializer for my class as > suggested in [3] because Flink will ignore my registered serializer as long > as it considers my type as *generic*. > > config.registerTypeWithKryoSerializer(classOf[MyClass], classOf[RadicalSerde]) > > > My question finally is: Do I need to provide this custom classes? Is there > any practical example for creating custom information like the above > mentioned? I have had a quick preliminary look at it but seems that I need > to provide a non-trivial amount of information to TypeInformation and > TypeSerializer interfaces. > > Thank you for your excellent work and help. > > Cheers. > > [1] - > https://ci.apache.org/projects/flink/flink-docs-stable/dev/custom_serializers.html > [2] - > http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Custom-Serializer-for-Avro-GenericRecord-td25433.html > [3] - > https://ci.apache.org/projects/flink/flink-docs-release-1.8/dev/types_serialization.html#defining-type-information-using-a-factory > -- > Andrea Spina > Head of R&D @ Radicalbit Srl > Via Giovanni Battista Pirelli 11, 20124, Milano - IT > > > -- *Andrea Spina* Head of R&D @ Radicalbit Srl Via Giovanni Battista Pirelli 11, 20124, Milano - IT
