gripleaf opened a new issue, #433: URL: https://github.com/apache/paimon-cpp/issues/433
### Search before asking - [x] I searched the existing issues for manifest partition pruning and found no matching issue. ### Motivation Partition-selective scans currently construct complete manifest entries and file metadata before discarding entries outside the requested partition. This also affects repeated reads from the bucket-scoped snapshot manifest cache, which can contain live files from many partitions. When a query needs only a small subset of those entries, decoding unrelated file metadata and statistics adds avoidable allocation and planning work. The optimization should apply consistently to ordinary manifest reads, manifest Arrow-cache reads, and exact snapshot-cache hits. ### Solution Evaluate partition predicates before constructing full `ManifestEntry` and `DataFileMeta` objects, reusing the existing manifest batch and row-range reading paths. On snapshot-cache hits, select the exact retained snapshot and skip other snapshots without materializing their entries. Preserve the existing lazy-decode option, ADD/DELETE merge semantics, remaining scan filters, retention limits, and metrics. Keep cached values complete across different partition predicates. Predicate evaluation errors must propagate; malformed snapshot-cache values should fall back to rebuilding from manifests. No table or cache format change is required. ### Anything else? Implementation: https://github.com/apache/paimon-cpp/pull/430 ### Are you willing to submit a PR? - [x] I'm willing to submit a PR (PR #430 is already open). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
