Hi Lasse,
are you using Avro specific records? A look into the code shows that the
warnings in the log are generated after the Avro check:
https://github.com/apache/flink/blob/master/flink-core/src/main/java/org/apache/flink/api/java/typeutils/TypeExtractor.java#L1741
Somehow your Avro object is not recognized correctly?
Regards,
Timo
On 16.07.20 13:28, Lasse Nedergaard wrote:
Hi.
We have some Avro objects and some of them contain the primitive data
type bytes and it's translated into java.nio.ByteBuffer in the Avro
objects. When using our Avro object we get these warnings:
org.apache.flink.api.java.typeutils.TypeExtractor [] - class
java.nio.ByteBuffer does not contain a getter for field hb
org.apache.flink.api.java.typeutils.TypeExtractor [] - class
java.nio.ByteBuffer does not contain a setter for field hb
org.apache.flink.api.java.typeutils.TypeExtractor [] - Class class
java.nio.ByteBuffer cannot be used as a POJO type because not all fields
are valid POJO fields, and must be processed as GenericType. Please read
the Flink documentation on "Data Types & Serialization" for details of
the effect on performance.
and it's correct that ByteBuffer doesn't contain getter and setter for hb.
Flink documentation said "Note that Flink is automatically serializing
POJOs generated by Avro with the Avro serializer.", but when I debug it
looks like it fails back to generic type for the byte buffer and it
therefore make sense with the warnings.
I want to ensure we are running as effectively as possible.
So my questions are:
1. What is the most optimal way to transport byte arrays in Avro in Flink.
2. Do Flink use Avro serializer for our Avro object when they contain
ByteBuffer?
Thanks
Lasse Nedergaard
