szehon-ho opened a new pull request, #55248: URL: https://github.com/apache/spark/pull/55248
### What changes were proposed in this pull request? Add tests to `DataSourceV2EnhancedPartitionFilterSuite` covering the case where `getPartitionFiltersAndDataFilters` (called from `PushDownUtils.pushPartitionPredicates`) extracts additional partition filters from predicates that reference both partition and data columns via `extractPredicatesWithinOutputSet`. Five new tests: 1. **Translated OR with mixed references** — `(part_col = 'a' AND data = 'x') OR (part_col = 'b' AND data = 'y')` extracts `part_col = 'a' OR part_col = 'b'` 2. **Untranslatable OR with mixed references** — same pattern with UDFs, goes directly to second pass 3. **OR with one partition-only and one mixed branch** — `part_col = 'a' OR (part_col = 'b' AND data = 'y')` extracts `part_col = 'a' OR part_col = 'b'` 4. **Multi-column extraction** — `(p1 = 'a' AND p2 = 'x' AND data = 'd1') OR (p1 = 'b' AND p2 = 'y' AND data = 'd4')` extracts `(p1 = 'a' AND p2 = 'x') OR (p1 = 'b' AND p2 = 'y')` 5. **Two partition predicates pushed** — UDF on p1 combined with extracted filter on p2 from a mixed OR, resulting in 2 separate `PartitionPredicate`s ### Why are the changes needed? The existing test suite did not cover the `extractPredicatesWithinOutputSet` code path where partition-only sub-expressions are inferred from mixed predicates (OR expressions referencing both partition and data columns). These tests ensure that partition pruning works correctly for these important query patterns. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? New unit tests added to `DataSourceV2EnhancedPartitionFilterSuite`. All 24 tests pass (19 existing + 5 new). ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude (Cursor) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
