alamb commented on code in PR #26634:
URL: https://github.com/apache/datafusion/pull/26634#discussion_r4568811714
##########
datafusion/physical-optimizer/src/projection_pushdown.rs:
##########
@@ -363,7 +374,28 @@ impl<'a> JoinFilterRewriter<'a> {
let depends_on_other_side =
self.depends_on_join_side(&expr, self.join_side.negate())?;
if depends_on_other_side || is_volatile(&expr) {
- return expr.map_children(|expr| self.rewrite(expr));
+ // The top-level conjuncts carry no evaluation order: the logical
optimizer splits the
Review Comment:
if we are going to handle conjuncts specially, let's split it before passing
down here - this looks very special cased to me
##########
datafusion/physical-optimizer/src/projection_pushdown.rs:
##########
@@ -363,7 +374,28 @@ impl<'a> JoinFilterRewriter<'a> {
let depends_on_other_side =
self.depends_on_join_side(&expr, self.join_side.negate())?;
if depends_on_other_side || is_volatile(&expr) {
- return expr.map_children(|expr| self.rewrite(expr));
+ // The top-level conjuncts carry no evaluation order: the logical
optimizer splits the
+ // filter into conjuncts, reorders them and may push single-side
ones below the join.
+ // Below them, children that the expression evaluates only for
some rows stay in the
+ // filter, because a projection below the join would evaluate them
for every row.
+ let conjunct = conjunct
Review Comment:
Can we extract this into a function or something?
##########
datafusion/physical-optimizer/src/projection_pushdown.rs:
##########
@@ -302,6 +306,10 @@ fn minimize_join_filter(
/// this is never the case, no subexpressions of the filter can be pushed
down. If there is a
/// subexpression that can be computed using only one side of the join, the
entire subexpression is
/// pushed down to the join side.
+///
+/// The top-level conjuncts of the filter are searched in full. Below them,
children that an
Review Comment:
this doesnt' make any sense to me -- can you please explain it in your own
words / with an example
It seems like what should happen is that the walk stops at the first
conditional expr it finds (aka it shouldn't walk into CASE exprs)
##########
datafusion/physical-optimizer/src/projection_pushdown.rs:
##########
@@ -433,6 +465,29 @@ impl<'a> JoinFilterRewriter<'a> {
}
}
+/// Returns the number of leading children of `expr` that are evaluated for
every row, or `None`
+/// if all children are. The remaining children may be skipped for some rows.
+///
+/// Mirrors `conditional_children` in the logical `CommonSubexprEliminate`
rule.
+fn always_evaluated_child_count(expr: &dyn PhysicalExpr) -> Option<usize> {
Review Comment:
If you only have this list in this function it can never cover all
expressions for which this holds (e.g. user defined physical expressions)
If you need this I think it needs to be a function on PhysicalExpr trait
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]