ump45nose opened a new pull request, #2642:
URL: https://github.com/apache/datafusion-ballista/pull/2642
# Which issue does this PR close?
Closes #2601.
# Rationale for this change
`SelectJoinRule::optimize` handled every `JoinSelectionAction` inline inside
a `transform_up` closure, several levels deep, with the swap check, the
"downcast to `ExchangeExec` and broadcast" step and the `replace_children(...,
Recompute)` call copy-pasted across arms. This makes it hard to add or change a
join strategy.
# What changes are included in this PR?
A pure refactor of
`ballista/scheduler/src/state/aqe/optimizer_rule/join_selection.rs`, following
the plan in the issue. Behaviour is meant to be identical:
- `optimize` returns early when adaptive join is disabled and delegates each
node to `select_join`, which returns early for nodes that are not a resolved
`DynamicJoinSelectionExec` and then dispatches on the action.
- Each action has its own method: `late_collect_left`, `collect_left`,
`stage_build_side`, `repartition`, `hash` (`Sort` is still passed through).
- The nested `transform_up` body of the CollectLeft arm moved into
`collect_build_side`, using early returns instead of the three-way if/else.
- New helpers `should_swap(&HashJoinExec)`, `broadcast_exchange(plan, side)`
and a free `with_children(plan, left, right)`.
Plan ids are still taken in the same order, and the error messages are
unchanged.
`cargo test -p ballista-scheduler --lib` passes (474 passed, 2 ignored,
including the 28 join selection tests), and `cargo clippy -p ballista-scheduler
--all-targets -- -D warnings` and `cargo fmt --all` are clean.
# Are there any user-facing changes?
No.
---
This PR was prepared with AI assistance; I reviewed the change and ran the
tests above.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
