Swinky Mann created SPARK-34598: ----------------------------------- Summary: RewritePredicateSubquery Rule must not update Filters without subqueries Key: SPARK-34598 URL: https://issues.apache.org/jira/browse/SPARK-34598 Project: Spark Issue Type: Improvement Components: SQL Affects Versions: 3.1.1 Reporter: Swinky Mann
1. Currently RewritePredicateSubquery rule updates Filter node for queries without any subquery as well. This shouldn't happen. 2. Also `Filter(conditions.reduce(And), child)` in the rule might create a skewed expression tree even though original expression is balanced. {noformat} === Applying Rule org.apache.spark.sql.catalyst.optimizer.RewritePredicateSubquery === Project [a#0] Project [a#0] !+- Filter (((a#0 > 1) OR (b#1 > 2)) AND ((c#2 > 1) AND (d#3 > 2))) +- Filter ((((a#0 > 1) OR (b#1 > 2)) AND (c#2 > 1)) AND (d#3 > 2)) +- LocalRelation <empty>, [a#0, b#1, c#2, d#3] +- LocalRelation <empty>, [a#0, b#1, c#2, d#3]{noformat} -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org