Xiao Li created SPARK-13840:
-------------------------------
Summary: Disable Project Push Down Through Filter
Key: SPARK-13840
URL: https://issues.apache.org/jira/browse/SPARK-13840
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 2.0.0
Reporter: Xiao Li
Priority: Critical
Now, ColumnPruning and PushPredicateThroughProject reverse each other's effect.
Optimizer rules always reach the max iteration when optimizing some queries.
Thus, we should disable columnPruning when hitting Filter for enabling
predicate push down through project.
The issue becomes worse when having another rule NullFiltering, which add extra
Filters.
In addition, columnPruning should not push Project through non-deterministic
Filters.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
