XiaoHongbo-Hope opened a new pull request, #1196: URL: https://github.com/apache/paimon-rust/pull/1196
A repeated ROW point read currently fetches and parses the footer and block index on every query against the same immutable file. With a one-block fixture, that makes three range reads per query (footer, index, block). Cache the parsed footer and block index within the FileIO metadata-cache context. The second read of the same file then needs only the data-block read. The cache is enabled by default when `file-format.metadata-cache.max-size` is nonzero, bounded to 32 MiB and 4,096 entries, keyed by the storage-namespaced immutable file identity and size, and bypassed when the reader has no stable key or after fork. No new option or file-format change is introduced. This is separate from #1195, which caches the ROW sidecar file size (`stat`); the two optimizations are independent. Both PRs touch the FileIO cache context, so this branch will need a trivial rebase if #1195 merges first. Validation: - Counted ROW reader test: first query reads footer/index/block (3); repeated query reads only block (1); disabled cache and readers without stable keys still perform all reads. - `cargo test --locked -p paimon --features storage-memory row_reader_ --lib` (7 passed) - `cargo test --locked -p paimon --features storage-memory row_metadata_cache_has_separate_type_and_bypasses_inherited_fork_state --lib` (1 passed) - `cargo test --locked -p paimon --test data_evolution_row_sidecar_test` (14 passed) - `cargo clippy --locked -p paimon --lib --features storage-memory -- -D warnings -A clippy::nonminimal_bool` (passed; the suppressed lint is present in unrelated main-branch files) - `cargo fmt --all`; `git diff --check` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
