mbutrovich opened a new pull request, #20699:
URL: https://github.com/apache/datafusion/pull/20699
## Which issue does this PR close?
Backport of #20482 to branch-52.
## Rationale for this change
Cherry-pick fix and prerequisites so that SortMergeJoin emits output
incrementally instead of waiting for all input to
complete. This resolves OOM issues Comet is seeing with DataFusion 52.
## What changes are included in this PR?
Cherry-picks of the following commits from `main`:
1. #19614 — Extract sort-merge join filter logic into separate module
2. #20463 — Use zero-copy slice instead of take kernel in sort merge join
3. #20482 — Fix SortMergeJoin to not wait for all input before emitting
## Are these changes tested?
Yes, covered by existing and new tests included in #20482.
## Are there any user-facing changes?
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
