[
https://issues.apache.org/jira/browse/SPARK-38148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17489211#comment-17489211
]
Apache Spark commented on SPARK-38148:
--------------------------------------
User 'ulysses-you' has created a pull request for this issue:
https://github.com/apache/spark/pull/35451
> Do not add dynamic partition pruning if there exists static partition pruning
> -----------------------------------------------------------------------------
>
> Key: SPARK-38148
> URL: https://issues.apache.org/jira/browse/SPARK-38148
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 3.3.0
> Reporter: XiDuo You
> Priority: Major
>
> Dynamic partition pruning add a filter as long as the join condition contains
> partition columns. But if there exists other condition which contains the
> static partition pruning, it's unnecessary to add an extra dynamic partition
> pruning.
> For example:
> {code:java}
> CREATE TABLE t1 (c1 int) USING PARQUET PARTITIONED BY (p1 string);
> CREATE TABLE t2 (c2 int) USING PARQUET PARTITIONED BY (p2 string);
> SELECT * FROM t1 JOIN t2 ON t1.p1 = t2.p2 and t1.p1 = 'a' AND t2.c2 > 0;
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
