viirya opened a new pull request, #59450: URL: https://github.com/apache/spark/pull/59450
### What changes were proposed in this pull request? Backport of #59334 to branch-3.5. `Decimal.fromString` and `Decimal.fromStringANSI` fast-fail on strings whose number of integral digits exceeds `DecimalType.MAX_PRECISION`, computed as `precision - scale` of the parsed `java.math.BigDecimal`. This PR: - computes that value as a `Long` instead of an `Int`; - skips the check for zero, which fits any decimal type whatever its exponent, and drops a negative scale of zero first so that the result has precision 1 and scale 0 instead of a precision derived from the exponent; - moves the check into one helper shared by `fromString` and `fromStringANSI`. The cherry-pick conflicted only on formatting: this branch predates the `sql/api` scalafmt reformatting, so the continuation indent of the old condition differed. The new helper keeps this branch's 4-space continuation indent. In the new `DecimalSuite` test, `checkError` takes `errorClass` instead of `condition` on this branch. ### Why are the changes needed? For strings with an exponent near `Int.MaxValue`, e.g. `'1e2147483647'` (precision 1, scale -2147483647), `precision - scale` overflows `Int` to a negative number and the fast-fail check is skipped. The subsequent conversion then fails with a raw `java.lang.ArithmeticException` (`BigInteger would overflow supported range`), so `try_cast`, legacy `CAST` and ANSI `CAST` all fail the task instead of returning NULL or raising `NUMERIC_OUT_OF_SUPPORTED_RANGE`. The check also counts integral digits for zero, so e.g. `'0e39'` is rejected as out of range although it fits any decimal type, while `'0e37'` returns `0.00`. ### Does this PR introduce _any_ user-facing change? Yes. Casting such strings to decimal now returns NULL (`try_cast` / non-ANSI `CAST`) or raises `NUMERIC_OUT_OF_SUPPORTED_RANGE` (ANSI `CAST`), the same as other out-of-range strings such as `'6E+38'`, instead of failing with `java.lang.ArithmeticException`. Zero with a large exponent, e.g. `'0e39'`, now returns `0` (precision 1, scale 0) instead of NULL / `NUMERIC_OUT_OF_SUPPORTED_RANGE`. ### How was this patch tested? The tests from #59334 were ported to `DecimalSuite`, `CastSuiteBase`, `CastWithAnsiOnSuite` and `CastWithAnsiOffSuite` (and run through `TryCastSuite`). On this branch with JDK 8, these four suites and `TryCastSuite` pass, and scalastyle passes for `sql-api` and `catalyst`. Without the fix (with `numDigitsInIntegralPart` reverted to `Int` arithmetic), the four new non-zero cases fail. ``` build/sbt 'catalyst/testOnly *DecimalSuite *CastWithAnsiOnSuite *CastWithAnsiOffSuite *TryCastSuite' ``` ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code (Claude Opus 5.5) This pull request and its description were written by Isaac. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
