voonhous commented on issue #20630:
URL: https://github.com/apache/hudi/issues/20630#issuecomment-6979161174

   ### Background: why `HudiAvroSerializer` exists
   
   hudi-common's file-group reader merges records of an engine type `T` through 
a `RecordContext<T>`. Spark, Flink and Hive each bring their own `T`. The Trino 
connector reuses `IndexedRecord` with the stock `AvroRecordContext`, so every 
row is converted from a page to an Avro record and back.
   
   The serializer is not a Hive-to-Avro type mapper. The Trino side is typed 
from the metastore and the Avro side from the table schema. `serialize()` puts 
`Type.getObjectValue(...)` into the Avro field with the same name without 
consulting the field's Avro type. So one field can hold a Trino-shaped value 
(base rows) or an Avro-native one (Avro log blocks), and `buildRecordInPage()` 
tells them apart with `instanceof`.
   
   Section 4 lists what a `TrinoRow` record type (the proposal above) removes. 
The numbers map each row to the part of today's path it replaces.
   
   <!-- diagram: why-serializer.png -->
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to