princejain756 opened a new pull request, #26636:
URL: https://github.com/apache/datafusion/pull/26636
## Which issue does this PR close?
- Closes #26625.
## Rationale for this change
A valid query failed with an internal error:
```sql
SELECT a AS x, a AS y FROM t
UNION ALL
SELECT a AS x, a AS y FROM t
ORDER BY x, y;
```
`SanityCheckPlan` rejected the plan because `SortPreservingMergeExec`
requires ordering `[x ASC, y ASC]` while `UnionExec` only reported `[x ASC]`.
Since `x` and `y` alias the same column, every input satisfies `x = y`, so `[x
ASC]` should satisfy the requirement. `UnionExec` dropped that fact:
`calculate_union_binary` rebuilt output `EquivalenceProperties` with constants
and orderings but no equivalence classes, so the `x = y` class present in each
child never reached the union output.
## What changes are included in this PR?
- `calculate_union_binary` now computes the meet of the inputs' equivalence
classes: each lhs class is partitioned by rhs equivalence, and only subsets
whose members are equivalent in both inputs are kept. Classes that hold in
every union input are added to the output properties alongside the existing
constants and orderings.
- New helper `partition_class_by_equivalence` implements the partitioning.
- Two unit tests cover full class preservation (including the
`ordering_satisfy_requirement` check from the issue) and partial overlap
(`{a,b,c}` vs `{a,b},{c,d}` yields only `{a,b}`).
- A sqllogictest case in `union.slt` reproduces the reported query end to
end.
## What is the testing strategy for this PR?
- `test_union_equivalence_properties_preserves_classes` and
`test_union_equivalence_properties_classes_partial_overlap` in
`datafusion/physical-expr/src/equivalence/properties/union.rs` fail without the
change and pass with it.
- The new `union.slt` query fails with `SanityCheckPlan` before the fix and
returns 4 sorted rows after.
- Ran: `cargo test -p datafusion-physical-expr` (1713 passed), `cargo test
-p datafusion-sqllogictest --test sqllogictests -- union` (5/5 files), `cargo
fmt --all -- --check`, and the CI clippy command `cargo clippy --all-targets
--workspace --features "avro,integration-tests,extended_tests" -- -D warnings`
(clean).
- Note: `cargo clippy --all-targets --all-features` currently fails on
`main` at `datafusion/core/src/dataframe/mod.rs` (`mutable_key_type`, unrelated
to this PR and reproducible without it).
## Are there any user-facing changes?
Yes: the query form from #26625 no longer fails planning. No API changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
