stevomitric commented on code in PR #58855:
URL: https://github.com/apache/spark/pull/58855#discussion_r4035533299
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala:
##########
@@ -991,6 +991,61 @@ object DateTimeUtils extends SparkDateTimeUtils {
}
}
+ /**
+ * Adds the specified number of units to a nanosecond-precision timestamp of
precision `p` (in
+ * [7, 9]), returning a value floored to that precision.
+ *
+ * For units of MICROSECOND or coarser the addition runs on the microsecond
grid (reusing
+ * [[timestampAdd]] for all the calendar, DST and overflow handling) and the
input's already
+ * `p`-aligned fraction is carried through unchanged. For the NANOSECOND
unit the fraction absorbs
+ * `quantity` nanoseconds and any whole microseconds carry into
`epochMicros`; the resulting
+ * fraction is then floored to `p` (via
[[truncateTimestampNanosToPrecision]]) so a NANOSECOND
+ * quantity finer than the type's step never produces an off-grid value that
would compare, hash
+ * or sort unequal to its displayed (aligned) form. NANOSECOND is only valid
here: adding
+ * nanoseconds to a microsecond timestamp is unrepresentable, so that
combination is rejected as
+ * an invalid unit through the microsecond [[timestampAdd]] path.
+ *
+ * @param unit A keyword that specifies the interval units to add to the
input timestamp.
+ * @param quantity The amount of `unit`s to add. It can be positive or
negative.
+ * @param ts The input nanosecond-precision timestamp.
+ * @param precision The declared fractional-second precision `p` in [7, 9]
of the input/output.
+ * @param zoneId The time zone ID at which the operation is performed.
+ * @return A nanosecond-precision timestamp value floored to `precision`.
+ */
+ def timestampAddNanos(
+ unit: String,
+ quantity: Long,
+ ts: TimestampNanosVal,
+ precision: Int,
+ zoneId: ZoneId): TimestampNanosVal = {
+ if (unit.toUpperCase(Locale.ROOT) == "NANOSECOND") {
+ try {
+ // Split the added nanoseconds into whole microseconds and a [0, 999]
remainder first, so
+ // the fraction sum stays within [0, 1998] and only the true
microsecond total can overflow
+ // a Long (adding the remainder to `nanosWithinMicro` before the split
would spuriously
+ // reject a large-but-representable quantity).
+ val quotientMicros = Math.floorDiv(quantity, NANOS_PER_MICROS)
+ val remainderNanos = Math.floorMod(quantity, NANOS_PER_MICROS)
+ val fractionSum = ts.nanosWithinMicro.toLong + remainderNanos
+ val carryMicros =
+ Math.addExact(quotientMicros, Math.floorDiv(fractionSum,
NANOS_PER_MICROS))
+ val newFraction = Math.floorMod(fractionSum, NANOS_PER_MICROS).toShort
+ val newMicros = Math.addExact(ts.epochMicros, carryMicros)
+ truncateTimestampNanosToPrecision(
+ TimestampNanosVal.fromParts(newMicros, newFraction), precision)
+ } catch {
+ case _: ArithmeticException | _: DateTimeException =>
+ throw QueryExecutionErrors.timestampAddOverflowError(ts.epochMicros,
quantity, unit)
Review Comment:
this only happens at the very edge of the Long range (~year 294247). I
reused the existing error instead of adding a nanosecond-specific one (the
digits are correct up to nanosecond), but I can add one if you'd like.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
