[
https://issues.apache.org/jira/browse/SPARK-13840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15192081#comment-15192081
]
Apache Spark commented on SPARK-13840:
--------------------------------------
User 'gatorsmile' has created a pull request for this issue:
https://github.com/apache/spark/pull/11682
> Disable Project Push Down Through Filter
> ----------------------------------------
>
> Key: SPARK-13840
> URL: https://issues.apache.org/jira/browse/SPARK-13840
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.0.0
> Reporter: Xiao Li
> Priority: Critical
>
> Now, ColumnPruning and PushPredicateThroughProject reverse each other's
> effect. Optimizer rules always reach the max iteration when optimizing some
> queries. Thus, we should disable columnPruning when hitting Filter for
> enabling predicate push down through project.
> The issue becomes worse when having another rule NullFiltering, which add
> extra Filters.
> In addition, columnPruning should not push Project through non-deterministic
> Filters.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
