[ https://issues.apache.org/jira/browse/SPARK-6715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14746442#comment-14746442 ]
Josh Rosen commented on SPARK-6715: ----------------------------------- Per discussion on the PR, I believe that this is "Won't Fix" for 1.x? > Eliminate duplicate filters from pushdown predicates > ---------------------------------------------------- > > Key: SPARK-6715 > URL: https://issues.apache.org/jira/browse/SPARK-6715 > Project: Spark > Issue Type: Improvement > Components: SQL > Reporter: Liang-Chi Hsieh > Priority: Minor > > Now in {{DataSourceStrategy}}, the pushdown predicates are duplicate of > original {{Filter}} conditions. Thus, some predicates are performed both by > data source relation and {{Filter}} plan. I think it is a duplicate loading. > Once the predicates are pushed down and performed by a data source relation, > it > can be eliminated from outside {{Filter}} plan's condition. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org