ranflarion opened a new pull request, #26668: URL: https://github.com/apache/datafusion/pull/26668
## Which issue does this PR close? - Closes #26667. ## Rationale for this change A table that declares a narrower element struct under a `FixedSizeList`, `ListView`, `LargeListView` or `Dictionary` column still reads every Parquet leaf. The read plan derives the emitted type with `type_for_leaf_subset`, which rejects partial selections under those wrappers, so the clip #24315 computes for them always falls back to a full read. ## What changes are included in this PR? `project_type_for_leaves` descends through the same wrappers as `clip_type`, rebuilding each around its projected child with the physical outer parameters (`FixedSizeList` keeps its size, `Dictionary` its key type). That is the type #24315's reader tests pin for those wrappers. ## What is the testing strategy for this PR? `type_for_leaf_subset_matches_clip_through_wrappers` checks that `type_for_leaf_subset` reproduces `clip_for_cast`'s type for every wrapper the clip descends through; on main it fails at `ListView`. `type_for_leaf_subset_rejects_unsupported_partial_wrapper` now uses `RunEndEncoded`, which stays opaque. A `FixedSizeList` section in `parquet_nested_schema_pruning.slt` pins `bytes_scanned` for the narrow table at 83 B against 174 B for the full schema; on main both read 174 B. `ListView` and `Dictionary` columns can't be declared in SQL, so the unit test is what covers them. ## Are there any user-facing changes? No API changes. Scans of such columns read fewer bytes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
