stevomitric opened a new pull request, #57680: URL: https://github.com/apache/spark/pull/57680
### What changes were proposed in this pull request? `SubtractTimestamps` (the `timestamp - timestamp` operator) previously only accepted the microsecond timestamp types (`AnyTimestampType`). This PR extends it to also accept the nanosecond-precision timestamp types (`TIMESTAMP_NTZ(p)` / `TIMESTAMP_LTZ(p)`, `p` in [7, 9]). The difference is always reported on the microsecond grid, since the result is a `DayTimeIntervalType` (which has microsecond resolution). Each operand therefore contributes only its `epochMicros`; the sub-microsecond remainder is truncated. This keeps `ts_nanos - ts_nanos` numerically identical to `ts_nanos::timestamp - ts_nanos::timestamp`. Concretely: - `SubtractTimestamps.inputTypes` widens to `TypeCollection(AnyTimestampType, AnyTimestampNanoType)` for both operands, and `nullSafeEval` / `doGenCode` reduce a `TimestampNanosVal` operand to its `epochMicros` before subtracting. - `BinaryArithmeticWithDatetimeResolver` recognizes a nanos operand on either side of a `Subtract` and routes it to `SubtractTimestamps`. - The `DateTimeOperations` / `AnsiDateTimeOperations` type-coercion rules unify mixed operands (DATE vs timestamp, or two timestamps differing in precision or time-zone family). A cross-family pair unifies in the NTZ family (mirroring the microsecond `TIMESTAMP - TIMESTAMP_NTZ` precedent); a same-family pair keeps its family so an LTZ pair still subtracts in the session time zone. For pure-microsecond inputs the behavior is unchanged. ### Why are the changes needed? Sub-task of SPARK-56822 (nanosecond-precision timestamp support). Without this, subtracting two nanosecond timestamps fails analysis. ### Does this PR introduce _any_ user-facing change? Yes. `timestamp_nanos - timestamp_nanos` (and mixed micro/nanos, and nanos/DATE) now produces a microsecond-grid `DayTimeIntervalType` instead of failing analysis. ### How was this patch tested? - New unit test in `DateExpressionsSuite` covering NTZ/LTZ nanos operands, sub-microsecond truncation, pre-epoch values, NULLs, and interpreted/codegen consistency. - New coercion cases in `TypeCoercionSuite` / `AnsiTypeCoercionSuite`. - New golden SQL cases in `timestamp-ntz-nanos.sql` / `timestamp-ltz-nanos.sql`. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code (Claude Opus 4.8) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
