voonhous opened a new pull request, #20757:
URL: https://github.com/apache/hudi/pull/20757

   ### Describe the issue this Pull Request addresses
   
   Follow-up to #20670: moves the connector's column types onto 
`TrinoTypeMapper`, behind a property because it changes what Trino reports for 
existing tables. It is also the base for #20630, which drops the per-row Avro 
conversion in MOR merges.
   
   ### Summary and Changelog
   
   Adds `hudi.schema-column-types.enabled` (session property 
`schema_column_types_enabled`), off by default. With it on, data columns take 
their Trino type from the table's `HoodieSchema` instead of the metastore's 
Hive type strings, so UUID, TIME, TIMESTAMP and VECTOR columns read as `uuid`, 
`time`, `timestamp [with time zone]` and arrays. With it off, nothing changes.
   
   - `HoodieSchemaAvroSerializer`: the merge path's serializer for these types, 
switching on `HoodieSchemaType` with no default branch.
   - `ParquetReadConversions`: reads UUID (stored as text) and VECTOR (raw 
little-endian bytes) as `varbinary` and converts them.
   - Record-level and secondary index keys, and Parquet stats domains, handle 
the new types.
   
   Not covered yet: stats and MDT column-stats skipping for timestamp with time 
zone and time, and partition columns, which stay metastore-typed.
   
   ### Impact
   
   None by default. With the property on, column types change for the types 
above.
   
   ### Risk Level
   
   low: gated, off by default. A MOR fixture with one column per `HoodieSchema` 
type checks `DESCRIBE`, snapshot and read-optimized reads with the property on.
   
   ### Documentation Update
   
   none until the default flips.
   
   ### Contributor's checklist
   
   - [x] Read through [contributor's 
guide](https://hudi.apache.org/contribute/how-to-contribute)
   - [x] Enough context is provided in the sections above
   - [x] Adequate tests were added if applicable
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to