[jira] [Issue Comment Deleted] (SPARK-28155) Improve SQL optimizer's predicate pushdown performance for cascading joins
[ https://issues.apache.org/jira/browse/SPARK-28155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-28155: - Comment: was deleted (was: User 'yeshengm' has created a pull request for this issue: https://github.com/apache/spark/pull/24956) > Improve SQL optimizer's predicate pushdown performance for cascading joins > -- > > Key: SPARK-28155 > URL: https://issues.apache.org/jira/browse/SPARK-28155 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 3.0.0 >Reporter: Yesheng Ma >Assignee: Yesheng Ma >Priority: Major > Fix For: 3.0.0 > > > The current catalyst optimizer's predicate pushdown is divided into two > separate rules: PushDownPredicate and PushThroughJoin. This is not efficient > for optimizing cascading joins such as TPC-DS q64, where a whole default > batch is re-executed just due to this. We need a more efficient approach to > pushdown predicate as much as possible in a single pass. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Issue Comment Deleted] (SPARK-28155) Improve SQL optimizer's predicate pushdown performance for cascading joins
[ https://issues.apache.org/jira/browse/SPARK-28155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-28155: - Comment: was deleted (was: This is committed with a wrong JIRA ID, `SPARK-28155`.) > Improve SQL optimizer's predicate pushdown performance for cascading joins > -- > > Key: SPARK-28155 > URL: https://issues.apache.org/jira/browse/SPARK-28155 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 3.0.0 >Reporter: Yesheng Ma >Assignee: Yesheng Ma >Priority: Major > Fix For: 3.0.0 > > > The current catalyst optimizer's predicate pushdown is divided into two > separate rules: PushDownPredicate and PushThroughJoin. This is not efficient > for optimizing cascading joins such as TPC-DS q64, where a whole default > batch is re-executed just due to this. We need a more efficient approach to > pushdown predicate as much as possible in a single pass. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
