stuhood opened a new pull request, #25304: URL: https://github.com/apache/datafusion/pull/25304
## Which issue does this PR close? - Closes #25301 ## Rationale for this change In `JoinSelection`, `try_collect_left` converts a hash join to `PartitionMode::CollectLeft` (broadcast) when either input is below configured byte/row thresholds. However, if the inputs are already co-partitioned on the join keys across multiple partitions (e.g. both partitioned by hash on the join keys or by range with matching boundaries), switching to `CollectLeft` introduces an unnecessary `CoalescePartitions` and broadcasts data across partitions or cluster nodes. Preserving `PartitionMode::Partitioned` when the inputs already satisfy the partitioned distribution requirements allows the join to execute locally within each partition with zero shuffles or network transfers. ## What changes are included in this PR? - Added `HashJoinExec::partitioned_input_distribution_requirements` and `HashJoinExec::inputs_satisfy_partitioned_requirements` in `datafusion/physical-plan/src/joins/hash_join/exec.rs`. - In `JoinSelection::try_collect_left` (`datafusion/physical-optimizer/src/join_selection.rs`), skip converting to `CollectLeft` if inputs already satisfy partitioned distribution requirements across >1 partition. ## What is the testing strategy for this PR? New unit tests. ## Are there any user-facing changes? No. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
