jayzhan211 opened a new pull request, #26643:
URL: https://github.com/apache/datafusion/pull/26643
## Which issue does this PR close?
- None. Related to #26570 and #26631, which also work on how equivalence
properties pass through operators.
## Rationale for this change
Operators that combine rows from several input partitions must forget the
facts that hold only inside each
partition: constants whose value may differ between partitions, and, unless
the operator keeps order, the input's
orderings. Today this rule is written out in four places, and the copies
disagree when the input has a single
partition:
| | One input partition | Several input partitions |
|---|---|---|
| `RepartitionExec` | keeps everything | drops per-partition constants, and
orderings unless order-preserving |
| join-key alignment in `EnforceDistribution` | keeps everything | drops
per-partition constants |
| `SortPreservingMergeExec` | drops per-partition constants | drops
per-partition constants |
| `CoalescePartitionsExec` | drops per-partition constants and orderings |
drops per-partition constants and orderings |
With one input partition, both `SortPreservingMergeExec` and
`CoalescePartitionsExec` pass the input stream
through, so nothing they drop has stopped being true.
## What changes are included in this PR?
- `EquivalenceProperties::merged(input_partitions, keeps_order)`: with one
input partition everything is kept;
otherwise per-partition constants are dropped, and orderings too unless
`keeps_order`.
- The four places above call it.
## What is the testing strategy for this PR?
- A unit test for `merged`: one partition against two, with and without
`keeps_order`.
- A test that `CoalescePartitionsExec` over one sorted partition reports
that ordering (it fails before this
change) and over two does not.
- Existing tests, including all sqllogictests, pass without changes; no plan
changes.
## Are there any user-facing changes?
`SortPreservingMergeExec` and `CoalescePartitionsExec` over a single input
partition keep the facts of their
input, so the planner may skip a sort it does not need. Results do not
change. New public method
`EquivalenceProperties::merged`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
