jiangxt2 opened a new pull request, #57372: URL: https://github.com/apache/spark/pull/57372
### What changes were proposed in this pull request? Add Long range validation for the Decimal-to-Timestamp cast path in both interpreted and codegen execution. When a Decimal value multiplied by `MICROS_PER_SECOND` (1,000,000) exceeds `Long` range, `BigDecimal.longValue()` silently truncates the low 64 bits per the Java specification. The fix detects this overflow and either throws `CAST_OVERFLOW` (ANSI mode) or returns `NULL` (non-ANSI mode). The cast implementation now validates the multiplication result against Long range before calling `longValue()`. `DecimalType → TimestampType` is also registered in `forceNullable` since the cast can now return null. ### Why are the changes needed? `CAST(DECIMAL(38,0) AS TIMESTAMP)` silently produces garbage timestamps when the Decimal value overflows Long after scaling. This affects both ANSI and non-ANSI modes. ```sql SET spark.sql.ansi.enabled=false; SELECT CAST(CAST(9999999999999 AS DECIMAL(20,0)) AS TIMESTAMP); -- Returns: -265697-05-04 09:44:49.448384 (garbage) -- Expected: NULL SET spark.sql.ansi.enabled=true; SELECT CAST(CAST(9999999999999 AS DECIMAL(20,0)) AS TIMESTAMP); -- Returns: -265697-05-04 09:44:49.448384 (garbage) -- Expected: CAST_OVERFLOW error ``` Root cause: `BigDecimal.longValue()` silently returns the low 64 bits when the value exceeds Long range. The cast implementation had no range checking. This is consistent with existing overflow protection: - The double-to-timestamp path guards against `NaN`/`Infinite` by returning null - `SecondsToTimestamp` uses `DATETIME_OVERFLOW` for similar overflow protection - SPARK-45816 fixed the same class of bug for timestamp-to-integer overflow ### Does this PR introduce _any_ user-facing change? Yes. Previously, `CAST(large_decimal AS TIMESTAMP)` returned a nonsensical timestamp value when the Decimal overflows Long after scaling. Now: - Non-ANSI mode: returns `NULL` (consistent with other overflow casts) - ANSI mode: throws `CAST_OVERFLOW` error This is a behavior change for an edge case that previously produced silently wrong results. ### How was this patch tested? Added unit tests in `CastWithAnsiOffSuite` and `CastWithAnsiOnSuite`: Non-ANSI mode (`CastWithAnsiOffSuite`): - Positive overflow: `Decimal(99999999999999999999, 38, 0)` → null - Negative overflow: `Decimal(-99999999999999999999, 38, 0)` → null - Scaled overflow: `Decimal(99999999999999999999.999999, 38, 6)` → null - Normal value: `Decimal(1, 10, 0)` → `MICROS_PER_SECOND` - Zero value: `Decimal(0, 10, 0)` → `0L` - Boundary just under: `Decimal(9223372036854, 20, 0)` → passes through - Boundary just over: `Decimal(9223372036855, 20, 0)` → null ANSI mode (`CastWithAnsiOnSuite`): - Positive overflow: throws `CAST_OVERFLOW` with correct error parameters - Negative overflow: throws `CAST_OVERFLOW` with correct error parameters ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code with Claude Opus 4.8 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
