wlhjason commented on code in PR #20597:
URL: https://github.com/apache/datafusion/pull/20597#discussion_r3293539199
##########
datafusion/substrait/src/logical_plan/producer/rel/join.rs:
##########
@@ -76,33 +55,28 @@ pub fn from_join(
left: Some(left),
right: Some(right),
r#type: join_type as i32,
- expression: join_expr,
+ expression: join_expression,
post_join_filter: None,
advanced_extension: None,
}))),
}))
}
fn to_substrait_join_expr(
- producer: &mut impl SubstraitProducer,
- join_conditions: &Vec<(Expr, Expr)>,
- eq_op: Operator,
- join_schema: &DFSchemaRef,
-) -> datafusion::common::Result<Option<Expression>> {
- // Only support AND conjunction for each binary expression in join
conditions
- let mut exprs: Vec<Expression> = vec![];
- for (left, right) in join_conditions {
- let l = producer.handle_expr(left, join_schema)?;
- let r = producer.handle_expr(right, join_schema)?;
- // AND with existing expression
- exprs.push(make_binary_op_scalar_func(producer, &l, &r, eq_op));
- }
-
- let join_expr: Option<Expression> =
- exprs.into_iter().reduce(|acc: Expression, e: Expression| {
- make_binary_op_scalar_func(producer, &acc, &e, Operator::And)
- });
- Ok(join_expr)
+ join_on: Vec<(Expr, Expr)>,
+ null_equality: NullEquality,
+ join_filter: Option<Expr>,
+) -> Option<Expr> {
+ // Combine join on and filter conditions into a single Boolean expression
(#7611)
+ let eq_op = match null_equality {
+ NullEquality::NullEqualsNothing => Operator::Eq,
+ NullEquality::NullEqualsNull => Operator::IsNotDistinctFrom,
+ };
+ let all_conditions = join_on
+ .into_iter()
+ .map(|(left, right)| binary_expr(left, eq_op, right))
+ .chain(join_filter);
Review Comment:
Regarding this comment from Claude:
> from_join behavioral change is observable on the wire
I don't think the behavior has actually changed here, as the new test shows
- we still get `(join_cond_1 AND join_cond_2) AND join_filter` rather than
`join_cond_1 AND (join_cond_2 AND join_filter)`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
