cshuo opened a new pull request, #19373: URL: https://github.com/apache/hudi/pull/19373
### Describe the issue this Pull Request addresses Closes #19372. Flink v10 metadata tables currently remain on the default storage layout, so metadata lookups continue to use the classic file-group reader even though native logs can be consumed as sorted LSM runs. ### Summary and Changelog - Persist `hoodie.table.storage.layout=lsm_tree` for newly created Flink metadata tables on table version 10 or later. - Preserve the existing persisted configuration for other engines and older table versions. - Use `HoodieLsmFileGroupReader` for LSM metadata-table slices containing only native logs. - Fall back to `HoodieFileGroupReader` for the default layout and slices containing legacy inline logs. - Disable reusable base/log reader caching on the LSM path while preserving the existing classic-reader reuse behavior. - Add Flink metadata-table coverage for the persisted layout, initial native HFile log, multi-log `files` partition lookups, and equivalent results with reuse enabled and disabled. ### Impact New Flink v10 metadata tables use the LSM storage layout and reader automatically. Existing metadata tables, older table versions, other engines, public APIs, and the classic metadata-reader cache behavior are unchanged. ### Risk Level Medium. This changes the storage layout and read path selected for newly created Flink v10 metadata tables. The change is isolated by the persisted layout and native-log checks, with a classic-reader fallback for legacy or mixed slices. Validation: - `TestHoodieBackedTableMetadataWriter#tableStorageLayoutForMetadataTable` - `TestStreamWriteOperatorCoordinator#testSyncMetadataTable` with Flink 2.1 - `TestHoodieBackedTableMetadata#testTableOperations` across reuse enabled/disabled and table versions 6, 8, and current ### Documentation Update None. This does not add a user-facing option or change the default layout of data tables; it changes the internal layout selected for newly created Flink v10 metadata tables. ### Contributor's checklist - [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [x] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
