alamb commented on code in PR #26634:
URL: https://github.com/apache/datafusion/pull/26634#discussion_r4568811714


##########
datafusion/physical-optimizer/src/projection_pushdown.rs:
##########
@@ -363,7 +374,28 @@ impl<'a> JoinFilterRewriter<'a> {
         let depends_on_other_side =
             self.depends_on_join_side(&expr, self.join_side.negate())?;
         if depends_on_other_side || is_volatile(&expr) {
-            return expr.map_children(|expr| self.rewrite(expr));
+            // The top-level conjuncts carry no evaluation order: the logical 
optimizer splits the

Review Comment:
   if we are going to handle conjuncts specially, let's split it before passing 
down here - this looks very special cased to me
   
   



##########
datafusion/physical-optimizer/src/projection_pushdown.rs:
##########
@@ -363,7 +374,28 @@ impl<'a> JoinFilterRewriter<'a> {
         let depends_on_other_side =
             self.depends_on_join_side(&expr, self.join_side.negate())?;
         if depends_on_other_side || is_volatile(&expr) {
-            return expr.map_children(|expr| self.rewrite(expr));
+            // The top-level conjuncts carry no evaluation order: the logical 
optimizer splits the
+            // filter into conjuncts, reorders them and may push single-side 
ones below the join.
+            // Below them, children that the expression evaluates only for 
some rows stay in the
+            // filter, because a projection below the join would evaluate them 
for every row.
+            let conjunct = conjunct

Review Comment:
   Can we extract this into a function or something?



##########
datafusion/physical-optimizer/src/projection_pushdown.rs:
##########
@@ -302,6 +306,10 @@ fn minimize_join_filter(
 /// this is never the case, no subexpressions of the filter can be pushed 
down. If there is a
 /// subexpression that can be computed using only one side of the join, the 
entire subexpression is
 /// pushed down to the join side.
+///
+/// The top-level conjuncts of the filter are searched in full. Below them, 
children that an

Review Comment:
   this doesnt' make any sense to me -- can you please explain it in your own 
words / with an example
   
   It seems like what should happen is that the walk stops at the first 
conditional expr it finds (aka it shouldn't walk into CASE exprs)



##########
datafusion/physical-optimizer/src/projection_pushdown.rs:
##########
@@ -433,6 +465,29 @@ impl<'a> JoinFilterRewriter<'a> {
     }
 }
 
+/// Returns the number of leading children of `expr` that are evaluated for 
every row, or `None`
+/// if all children are. The remaining children may be skipped for some rows.
+///
+/// Mirrors `conditional_children` in the logical `CommonSubexprEliminate` 
rule.
+fn always_evaluated_child_count(expr: &dyn PhysicalExpr) -> Option<usize> {

Review Comment:
   If you only have this list in this function it can never cover all 
expressions for which this holds (e.g. user defined physical expressions)
   
   If you need this I think it needs to be a function on PhysicalExpr trait
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to