Hello!
I have a custom Avro GenericRecord serializer that supports reading and writing
records without having to pass along the schema with every record by using a
centralized registry. I've registered it with the execution environment as:
environment.addDefaultKryoSerializer(GenericRecord.class,
CustomGenericRecordSerializer.class);
We read from a source that provides a Tuple2<Void, GenericRecord>, and then map
to GenericRecord to process and then sink. If I disable Kryo generic types
given I don't want GenericRecord to fall back to the plain Kryo serializer, I
encounter the following exception:
Caused by: java.lang.UnsupportedOperationException: Generic types have been
disabled in the ExecutionConfig and type org.apache.avro.generic.GenericRecord
is treated as a generic type.
at
org.apache.flink.api.java.typeutils.GenericTypeInfo.createSerializer(GenericTypeInfo.java:87)
at
org.apache.flink.api.java.typeutils.TupleTypeInfo.createSerializer(TupleTypeInfo.java:104)
at
org.apache.flink.api.java.typeutils.TupleTypeInfo.createSerializer(TupleTypeInfo.java:49)
at
org.apache.flink.optimizer.postpass.JavaApiPostPass.createSerializer(JavaApiPostPass.java:310)
at
org.apache.flink.optimizer.postpass.JavaApiPostPass.traverseChannel(JavaApiPostPass.java:270)
at
org.apache.flink.optimizer.postpass.JavaApiPostPass.traverse(JavaApiPostPass.java:96)
at
org.apache.flink.optimizer.postpass.JavaApiPostPass.postPass(JavaApiPostPass.java:81)
at org.apache.flink.optimizer.Optimizer.compile(Optimizer.java:543)
at org.apache.flink.optimizer.Optimizer.compile(Optimizer.java:404)
It seems the GenericRecord in the Tuple2 gets interpreted as a GenericTypeInfo.
I expected it should use the CustomGenericRecordSerializer that is registered.
Have I misunderstood how the serializer registration works? Is there an extra
step when using these types as part of a Tuple?
best,
ah
________________________________
Your Personal Data: We may collect and process information about you that may
be subject to data protection laws. For more information about how we use and
disclose your personal data, how we protect your information, our legal basis
to use your information, your rights and who you can contact, please refer to:
www.gs.com/privacy-notices<http://www.gs.com/privacy-notices>
