gene-bordegaray opened a new issue, #23478: URL: https://github.com/apache/datafusion/issues/23478
Related: - [#22395](https://github.com/apache/datafusion/issues/22395) - [#21992](https://github.com/apache/datafusion/issues/21992) - [#23184](https://github.com/apache/datafusion/pull/23184) - [#23451](https://github.com/apache/datafusion/issues/23451) - [#23266](https://github.com/apache/datafusion/issues/23266) ## Is your feature request related to a problem or challenge? #23451 makes `SortMergeJoinExec` require co-partitioned children, but compatible range-partitioned inputs still take the conservative hash-repartition path. A sort merge join can process matching partition indexes locally once the inputs have compatible range layouts. ## Example ```text SortMergeJoinExec: join_type=Inner, on=[(range_key@0, range_key@0)] SortExec: expr=[range_key@0 ASC], preserve_partitioning=[true] DataSourceExec: output_partitioning=Range([range_key@0 ASC], [(10), (20), (30)], 4) SortExec: expr=[range_key@0 ASC], preserve_partitioning=[true] DataSourceExec: output_partitioning=Range([range_key@0 ASC], [(10), (20), (30)], 4) ``` The compatible range inputs are already co-partitioned by partition index. The optimizer should not insert hash repartitions; the per-partition sorts remain necessary when the source does not provide ordering. ## Describe the solution you'd like Opt `SortMergeJoinExec` into the temporary compatible-range satisfaction path for co-partitioned inputs. Covered join types: - All supported `SortMergeJoinExec` join types that execute matching partition indexes locally. ## Acceptance criteria - Compatible range/range inputs avoid hash repartitioning for the covered join types. - Incompatible split points, partition counts, sort options, or key expressions still trigger repartitioning. - Tests cover inner, left, right, and full joins, including matched and unmatched rows. - The plan retains per-partition sorts when input ordering is not otherwise satisfied. ## Additional context This should follow the multi-child requirement model added in #23451 and precede the general consolidation in #23266. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
