voonhous commented on issue #20630: URL: https://github.com/apache/hudi/issues/20630#issuecomment-6979161174
### Background: why `HudiAvroSerializer` exists hudi-common's file-group reader merges records of an engine type `T` through a `RecordContext<T>`. Spark, Flink and Hive each bring their own `T`. The Trino connector reuses `IndexedRecord` with the stock `AvroRecordContext`, so every row is converted from a page to an Avro record and back. The serializer is not a Hive-to-Avro type mapper. The Trino side is typed from the metastore and the Avro side from the table schema. `serialize()` puts `Type.getObjectValue(...)` into the Avro field with the same name without consulting the field's Avro type. So one field can hold a Trino-shaped value (base rows) or an Avro-native one (Avro log blocks), and `buildRecordInPage()` tells them apart with `instanceof`. Section 4 lists what a `TrinoRow` record type (the proposal above) removes. The numbers map each row to the part of today's path it replaces. <!-- diagram: why-serializer.png --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
