fangchenli opened a new issue, #23461:
URL: https://github.com/apache/datafusion/issues/23461
### Describe the bug
When join reordering swaps a join's inputs, the swapped plan keeps a
different value for a conflicting schema-level metadata key than the original,
so OptimizationInvariantChecker rejects it:
Internal error: PhysicalOptimizer rule 'join_selection' failed. Schema
mismatch.
swap_inputs rebuilds the join with inputs reversed and wraps it in a
reverting ProjectionExec (reorder_output_after_swap), which restores column
order and field metadata but not the schema-level metadata. So reversing two
inputs that disagree on a key flips the winner, and the pre/post-swap schemas
differ. This is on main, independent of merge direction, and shared by cross /
hash / NLJ joins.
### To Reproduce
```rust
// l: 1000 rows, schema metadata {"pandas": "LEFT"}
// r: 2 rows, schema metadata {"pandas": "RIGHT"}
ctx.sql("SELECT * FROM l CROSS JOIN r").await?.collect().await?;
// -> Internal error: ... 'join_selection' failed. Schema mismatch.
```
set datafusion.optimizer.join_reordering = false; makes it pass.
### Expected behavior
A swap must preserve the output schema, including schema-level metadata.
### Additional context
Found while fixing the cross-join initial-plan case in #23434 / PR #23442,
which leaves the shared swap path untouched.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
