voonhous opened a new issue, #19249: URL: https://github.com/apache/hudi/issues/19249
Sub-task of #18780 (RFC-105 Trino connector migration). ### Context `HudiTrinoReaderContext.buildRequiredColumnHandles` resolves each field of the file-group reader's `requiredSchema` to a `HiveColumnHandle` so the on-demand native (RFC-103 parquet) delta-log page source reads the columns needed for the merge. It resolves against `colNameToHandle`, which is built only from the connector projection (`hudiMetaAndDataColumnHandles`) -- and that projection carries only the query columns plus `HUDI_REQUIRED_META_COLUMNS` (record key + partition path). Meta columns the merge needs but the query did not project -- e.g. `_hoodie_commit_time` -- miss the map. Those misses are currently patched by hand-constructing `new HiveColumnHandle(name, 0, HIVE_STRING, VARCHAR, ...)` inline. It is correct (all Hudi meta columns are UTF8 strings, and the index-0 placeholder is never read because the parquet read resolves columns by name), but it reads as arbitrary and duplicates the meta-column handle construction already in `HudiUtil.prependHudiMetaAndOrderingColumns`. ### Proposal Build `colNameToHandle` from the full data schema (all Hudi meta columns + all data columns, typed once from the table schema) so every `requiredSchema` field resolves by a plain map lookup. Then, in `buildRequiredColumnHandles`, delete both: - the inline `new HiveColumnHandle(...)` synthesis, and - the `HOODIE_META_COLUMNS` guard / `IllegalArgumentException`. ### Acceptance - `TestHudiCustomMerger` / `TestHudiCustomMergerEndToEnd` (native v10 MOR merge) still pass. - No behavioral change for base-file or classic-avro-log reads. Pointer: `hudi-trino/src/main/java/io/trino/plugin/hudi/reader/HudiTrinoReaderContext.java` -> `buildRequiredColumnHandles`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
