jiangxt2 opened a new pull request, #58029:
URL: https://github.com/apache/spark/pull/58029
### What changes were proposed in this pull request?
CAST(decimal AS TIMESTAMP) scales the decimal value by 1,000,000 and
truncates the result with BigDecimal.longValue(). When the scaled product
exceeds the Long range, longValue() silently drops the high bits and produces
an incorrect timestamp that still looks valid. This PR checks the exact
BigDecimal product against the closed interval [Long.MinValue, Long.MaxValue]
before narrowing, in both the interpreted and whole-stage codegen paths. On
overflow, non-ANSI mode returns NULL and ANSI mode raises CAST_OVERFLOW through
the existing cast overflow handling. In-range values keep the existing
truncation of sub-microsecond fractions. Decimal-to-Timestamp casts are marked
nullable through Cast.forceNullable so the plan reflects that the result can be
NULL.
### Why are the changes needed?
The current behavior silently returns incorrect data: for example,
CAST(99999999999999999999BD AS TIMESTAMP) previously returned a value that
looks like a valid timestamp, so the failure is invisible to users and hard to
trace in downstream pipelines.
### Does this PR introduce _any_ user-facing change?
Yes. For decimal values whose product with 1,000,000 exceeds the Long range,
CAST(decimal AS TIMESTAMP) now returns NULL in non-ANSI mode and raises
CAST_OVERFLOW in ANSI mode, instead of silently returning an incorrect
timestamp. The behavior change is documented in the "Upgrading from Spark SQL
4.3 to 4.4" section of the SQL migration guide.
### How was this patch tested?
New test cases were added in CastWithAnsiOffSuite and CastWithAnsiOnSuite
covering positive and negative overflow, the exact Long boundaries,
boundary-adjacent values with sub-microsecond fractions, and the CAST_OVERFLOW
error parameters. TryCastSuite asserts that try_cast(decimal AS TIMESTAMP) is
nullable. A new DecimalTimestampCastSuite covers SQL execution with whole-stage
codegen enabled and disabled. The 413 targeted Catalyst tests and the SQL core
execution tests pass. The Catalyst module full suite passed before the final
test additions and was not rerun afterward. The full sql/test module run
remains blocked by a timeout in an unmodified SQLQuerySuite test case ("run sql
directly on files"), which reproduces when the suite runs alone and does not
involve any code changed by this PR.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Codex and Claude AI
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
