andygrove opened a new issue, #5075: URL: https://github.com/apache/datafusion-comet/issues/5075
### Describe the bug `QueryPlanSerde` derives the `nullOnOverflow` flag for decimal-promotion `CheckOverflow` wrappers from the live session config instead of the per-expression eval mode: ```scala // spark/src/main/scala/org/apache/comet/serde/QueryPlanSerde.scala:787 val newExpr = DecimalPrecision.promote(expr, !SQLConf.get.ansiEnabled) ``` Spark 4.1 captures the eval mode per expression (`NumericEvalContext` on `Add`/`Subtract`/`Multiply`/`Divide`/`IntegralDivide`/`Remainder`/`Pmod`/`Sum`), precisely because the expression's mode and the session's current ANSI setting can disagree: views created under a different ANSI setting, or `try_` operations inside an ANSI session (`try_add` lowers to `Add(..., EvalMode.TRY)`, not a separate node). This is currently latent rather than live, because the reachable decimal paths all get the mode from a per-expression source that fires first: - overflow-capable decimal add/sub/mul takes the wide-decimal path, whose `CheckOverflow` is deliberately elided (`native/core/src/execution/planner.rs:519-526`) in favor of `WideDecimalBinaryExpr`'s own per-expression `eval_mode`; - decimal divide gets an inner `CheckOverflow` from `CometDivide` carrying `expr.evalMode == EvalMode.ANSI` (`spark/src/main/scala/org/apache/comet/serde/arithmetic.scala:279-288`), which fires before the outer promotion wrapper; - remainder result precision is always exact, so no overflow. But any future change to those paths turns this into a live wrong-mode bug, and the current code reads as if the session flag were the right source. ### Expected behavior The flag passed to `DecimalPrecision.promote` should come from the expression's own eval mode (via the shim accessors already used elsewhere, e.g. `CometEvalModeUtil`), not `SQLConf.get.ansiEnabled`. ### Additional context Found by an ANSI-mode audit of all native expressions against Spark 4.1.1. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
