Linhong Liu created SPARK-61572:
-----------------------------------
Summary: ANSI cast of a DECIMAL with precision > 18 to
TINYINT/SMALLINT/INT rounds via Double instead of truncating the exact value
Key: SPARK-61572
URL: https://issues.apache.org/jira/browse/SPARK-61572
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 5.0.0
Reporter: Linhong Liu
Decimal.roundToNumeric (used by roundToInt/Short/Byte) handles the compact
longVal case exactly, but for a BigDecimal-backed DECIMAL (precision > 18) it
does decimalVal.toDouble and then range-checks and truncates the Double. Going
through a Double rounds values such as 0.999999999999999999 up to 1 before
truncating, and makes values just inside the INT range (e.g.
2147483647.999999999999) overflow. roundToLong already avoids this via
toBigInteger.longValueExact().
Fix: truncate the exact value toward zero for the BigDecimal case
(toBigInteger) with an exact range check, in both the interpreted and codegen
ANSI cast paths. A new conf spark.sql.legacy.castDecimalToIntegralTypeViaDouble
(default false) restores the previous behavior. Found via correctness analysis.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]