voonhous opened a new pull request, #20649:
URL: https://github.com/apache/hudi/pull/20649

   ### Describe the issue this Pull Request addresses
   
   Closes #20628. Part of #20625. Stacked on the benchmark PR for #20626; only 
the top commit is new.
   
   Every merge-path split built its own meta client and resolved the table 
schema on the worker, reading `hoodie.properties` once and the timeline twice 
per split.
   
   ### Summary and Changelog
   
   - `HudiTableState`: table config plus, for table versions below 8, the 
committed instants; captured once on the coordinator.
   - `HudiTableHandle` carries it together with the lazily resolved schema for 
MOR tables.
   - `HudiPageSourceProvider` passes both to the file group reader and no 
longer creates a meta client per split.
   - The benchmark's `schema_in_handle` variant is removed, since it is now the 
only behavior.
   
   Local benchmark, scan `_rt`, 48 partitions, default injected latency:
   
   | | before | after |
   |---|---|---|
   | `hoodie.properties` reads | 50 | 2 |
   | timeline reads | 97 | 2 |
   | total requests | 723 | 533 |
   | wall | 4.22 s | 3.59 s |
   
   ### Impact
   
   Fewer storage requests per MOR split. Two behavior changes: the schema is 
resolved once per query on the coordinator rather than per split, and 
`hoodie.table.create.schema` is not shipped in the handle.
   
   ### Risk Level
   
   medium. The handle grows by the table config. Covered by handle 
serialization tests, a schema-change test, and v6/v8 snapshot read 
file-operation tests.
   
   ### Documentation Update
   
   none
   
   ### Contributor's checklist
   
   - [x] Read through [contributor's 
guide](https://hudi.apache.org/contribute/how-to-contribute)
   - [x] Enough context is provided in the sections above
   - [x] Adequate tests were added if applicable
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to