akshaytayal opened a new issue, #13436:
URL: https://github.com/apache/gluten/issues/13436
Part of #13181 (sub-theme #13178). Found while re-enabling the Spark 4.2 UTs
disabled in #13163.
### Problem
`GlutenDateExpressionsSuite` `TruncTimestamp of Long.MinValue overflows with
ArithmeticException` fails on Spark 4.2:
```
GlutenException: VeloxUserError ... Could not convert
Timestamp(-9223402780800, 0) to microseconds
Function: toMicros File: velox/type/Timestamp.h
(raised in UnsafeRowFast::serializeFixedWidth / VeloxColumnarToRow)
```
Spark 4.2 throws `ArithmeticException("long overflow")` from `date_trunc`
itself.
### Root cause
- Spark 4.2 `DateTimeUtils.truncTimestamp` / `truncToUnitFast` use
`Math.addExact` / `subtractExact`, so truncating below the int64-microsecond
range throws `long overflow`.
- Velox `sparksql::DateTruncFunction`
(`velox/functions/sparksql/DateTimeFunctions.h`) calls
`functions::truncateTimestamp` (`velox/functions/lib/TimeUtils.h`), which works
on (seconds, nanos) and returns an unrepresentable `Timestamp(-9223402780800,
0)` without error.
- The error only surfaces if the value is later converted to micros (row
conversion). If a native operator consumes it first (compare, filter, cast,
write), the result is silently out of range while Spark throws.
### Proposed fix
Velox, `DateTruncFunction::call`: after truncation, check that the result
fits in int64 microseconds and raise `VELOX_ARITHMETIC_ERROR("long overflow")`
otherwise; add a sparksql test for all `date_trunc` units with
`Timestamp::fromMicros(INT64_MIN)`.
Then in Gluten add a test variant that asserts the overflow error (wrapped
in `GlutenException`, message contains `long overflow`) and keep the original
excluded; or map Velox arithmetic errors to `java.lang.ArithmeticException`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
