andygrove opened a new issue, #5072:
URL: https://github.com/apache/datafusion-comet/issues/5072
### Describe the bug
Three native ANSI error paths raise raw Arrow errors instead of
`SparkError`, so they bypass the JSON error converter
(`SparkErrorConverter.convertToSparkException` bails when the message is not
JSON) and surface as generic `CometNativeException` /
`CometQueryExecutionException` instead of `SparkArithmeticException` with the
proper error class, SQLSTATE, and query context.
1. **Wide-decimal arithmetic overflow under ANSI**:
`native/spark-expr/src/wide_decimal_binary_expr.rs:343` returns
`ArrowError::ComputeError("Arithmetic overflow")`. Note that
`native/core/src/execution/planner.rs:519-526` deliberately skips the wrapping
`CheckOverflow` (which would produce the correct `NUMERIC_VALUE_OUT_OF_RANGE`)
when the child is a `WideDecimalBinaryExpr`, so this Arrow error is the only
signal the user gets.
2. **Decimal divide/remainder by zero under ANSI**:
`native/spark-expr/src/math_funcs/div.rs:96, 123` raises
`ArrowError::ComputeError(divide_by_zero_error().to_string())`. The message
text contains `[DIVIDE_BY_ZERO]` but the exception type is wrong. The existing
test (`CometExpressionSuite.scala:2984-2988`) only asserts on message text, so
it passes.
3. **Decimal-to-decimal cast overflow under ANSI**: no dedicated arm in
`cast_array`; the fall-through DataFusion cast with `safe=false` propagates
`ArrowError::InvalidArgumentError("... too large to store in a Decimal128 of
precision ...")` instead of Spark's
`[NUMERIC_VALUE_OUT_OF_RANGE.WITHOUT_SUGGESTION] ... cannot be represented as
Decimal(p, s)`. This divergence is explicitly special-cased in
`CometCastSuite.scala:2402-2405`; fixing it allows deleting that special case.
### Steps to reproduce
```sql
SET spark.sql.ansi.enabled=true;
-- decimal div by zero (correct message text, wrong exception type):
SELECT CAST(1 AS DECIMAL(10,2)) / CAST(0 AS DECIMAL(10,2));
-- decimal to decimal overflow (wrong class and message):
SELECT CAST(CAST(12345.678 AS DECIMAL(20,3)) AS DECIMAL(4,3));
```
### Expected behavior
All three paths raise `SparkError` variants (`NumericValueOutOfRange`,
`DivideByZero`) so the converter produces genuine Spark exceptions, as already
happens for e.g. integer overflow and array-index errors.
### Additional context
Found by an ANSI-mode audit of all native expressions against Spark 4.1.1.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
