voonhous opened a new issue, #20632: URL: https://github.com/apache/hudi/issues/20632
Part of #20625. Line references are against master `db90a0da`. ### Problem `HudiSplit` carries each log file's size, but `HudiUtil.convertToFileSlice` (`HudiUtil.java:547`) rebuilds each log as `new HoodieLogFile(path)` with size -1, so every later size lookup goes to storage. For table version 10 native log files (`<file>.log.parquet`) this means, per log file: several `getPathInfo` sequences (`directoryExists` + `exists` + `length` + `lastModified`), a footer read for the block header, and a second footer read plus a length probe when the connector's page source opens the file. A base-only split pays one footer read and no probes. ### Direction Find the calls that repeat `length()`/`exists()`/listing on the same log file (the native log reader, `FSUtils.getFileSize`, `HudiTrinoStorage.getPathInfo`) and reuse the size and status already known from the file slice. ### Exit criteria - `HudiMorReadBenchmark` scan with `-Dbench.nativeLogs=true`: <= 1 HEAD and 0 LIST per log file (total scan `_rt` HEAD <= number of base + log files; LIST limited to partition and timeline listings). - Scan `_rt` wall time with native logs within 1.3x of the legacy-log run at the same latency profile. - Correctness asserts pass with native logs; existing native-log tests green. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
