[
https://issues.apache.org/jira/browse/SPARK-32903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-32903:
------------------------------------
Assignee: Apache Spark (was: L. C. Hsieh)
> GeneratePredicate should be able to eliminate common sub-expressions
> --------------------------------------------------------------------
>
> Key: SPARK-32903
> URL: https://issues.apache.org/jira/browse/SPARK-32903
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 3.1.0
> Reporter: L. C. Hsieh
> Assignee: Apache Spark
> Priority: Major
>
> Both {{GenerateMutableProjection}} and {{GenerateUnsafeProjection}}, such
> codegen objects can eliminate common sub-expressions. But
> {{GeneratePredicate}} currently doesn't do it.
> We encounter a customer issue that a Filter pushed down through a Project
> causes performance issue, compared with not pushed down case. The issue is
> one expression used in Filter predicates are run many times. Due to the
> complex schema, the query nodes are not wholestage codegen, so it runs
> {{Filter.doExecute}} and then call {{GeneratePredicate}}. The common
> expression was run many time and became performance bottleneck.
> {{GeneratePredicate}} should be able to eliminate common sub-expressions for
> such case.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
