L. C. Hsieh created SPARK-61663:
-----------------------------------
Summary: Let each expression declare whether it is total instead
of the allowlist in ExprUtils.isTotalOperator
Key: SPARK-61663
URL: https://issues.apache.org/jira/browse/SPARK-61663
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 5.0.0
Reporter: L. C. Hsieh
ExprUtils.isTotalOperator decides whether an expression can raise an error by
itself with one hand-maintained pattern match over expression classes.
ExprUtils.canEvaluateUnconditionally, ConstantPropagation and the optimizer and
execution rules that call them all go through it.
Two things make the list hard to maintain:
* An expression that is missing from the list is only left out of some
optimizations, so nothing flags the omission, and the list lives far from the
expression it describes.
* The cases for Cast and the arithmetic expressions depend on the input types
and the eval mode, and their results rely on the order of the case clauses.
This proposes to let each expression declare it:
* Expression.isTotal is the declaration, false by default.
* An expression that is always total mixes in the existing marker trait
NoThrow, which fixes isTotal to true.
* Cast, Add, Subtract, Multiply, Divide, IntegralDivide, Remainder, Pmod,
UnaryMinus and Abs override isTotal by their input types and eval mode, each as
a union of independent conditions.
* ExprUtils.isTotalOperator(e) becomes e.isTotal, and its callers do not change.
This is a refactoring with no behavior change: the set of expressions
isTotalOperator accepts stays the same in every eval mode.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]