L. C. Hsieh created SPARK-61556:
-----------------------------------
Summary: Generate FilterExec predicate code with and without
subexpression elimination from one implementation
Key: SPARK-61556
URL: https://issues.apache.org/jira/browse/SPARK-61556
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 4.4.0
Reporter: L. C. Hsieh
{{FilterExec.doConsume}} re-implements, for the conjunctions it eliminates
common subexpressions from, the algorithm of
{{GeneratePredicateHelper.generatePredicateCode}} that orders the conjuncts and
the IsNotNull checks in the generated code. The two copies are kept in sync
only by comments ("Any change to this algorithm must be mirrored there (and
vice versa)"), and earlier fixes such as SPARK-51356 and SPARK-60315 had to be
applied to both.
This is to have a single implementation: {{generatePredicateCode}} takes the
states of the common subexpressions, evaluates each right before the first
conjunct that references it, after the IsNotNull checks of that conjunct, and
generates the checks without the states. {{FilterExec.doConsume}} keeps the
decision to eliminate, the evaluation of the input variables and the creation
of the states.
No behavior change is intended: the generated code without subexpression
elimination stays the same, and with it the code differs in whitespace only.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]