marcuslin123 opened a new pull request, #59441: URL: https://github.com/apache/spark/pull/59441
### What changes were proposed in this pull request? This PR bounds decimal plain-string formatting when a decimal has a huge negative scale. `Decimal.toPlainString` continues to use plain notation through 1,000 implied trailing zeros, but falls back to scientific notation for larger values. The PR also adds regression coverage for the `Decimal` boundary, ANSI decimal-to-string casts in interpreted and code-generated evaluation, and `ToPrettyString`, which is used by `Dataset.show`. ### Why are the changes needed? When `spark.sql.legacy.allowNegativeScaleOfDecimal` is enabled, a decimal can have a scale as low as `Int.MinValue`. Calling `BigDecimal.toPlainString` for a value such as `1E+2147483647` attempts to materialize billions of trailing zeros and fails with an `OutOfMemoryError`. Centralizing the limit in `Decimal.toPlainString` protects all existing callers and avoids duplicating the scale check in interpreted casts, generated casts, and pretty-string formatting. ### Does this PR introduce _any_ user-facing change? Yes. Decimals with more than 1,000 trailing zeros implied by a negative scale are rendered in scientific notation instead of plain notation when cast to string in ANSI mode or displayed with `Dataset.show`. Smaller decimal values retain their existing formatting. ### How was this patch tested? Added regression tests to `DecimalSuite`, `CastWithAnsiOnSuite`, and `ToPrettyStringSuite`. ```bash build/sbt 'catalyst/testOnly org.apache.spark.sql.types.DecimalSuite org.apache.spark.sql.catalyst.expressions.CastWithAnsiOnSuite org.apache.spark.sql.catalyst.expressions.ToPrettyStringSuite' build/sbt sql-api/scalastyle catalyst/scalastyle catalyst/Test/scalastyle ``` ### Was this patch authored or co-authored using generative AI tooling? Generated-by: OpenAI Codex (GPT-5), used for code assistance under the author's direction and review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
