[
https://issues.apache.org/jira/browse/SPARK-60743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Max Gekk updated SPARK-60743:
-----------------------------
Affects Version/s: 3.3.0
4.1.0
4.2.0
3.5.0
4.3.0
3.2.0
4.0.0
3.4.0
> make_timestamp silently wraps an out-of-range seconds value
> -----------------------------------------------------------
>
> Key: SPARK-60743
> URL: https://issues.apache.org/jira/browse/SPARK-60743
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 3.2.0, 3.3.0, 3.4.0, 3.5.0, 4.1.0, 4.0.0, 4.2.0, 4.3.0,
> 5.0.0
> Reporter: Max Gekk
> Priority: Major
> Labels: pull-request-available
>
> make_timestamp and make_timestamp_ntz take their seconds argument as
> DECIMAL(16, 6), but DateTimeExpressionUtils.makeTimestampMicros narrows its
> unscaled micros to an int:
> // 8 digits cannot overflow Int.
> int totalMicros = (int) secAndMicros.toUnscaledLong();
> The comment was true while the input was DECIMAL(8, 6). Since SPARK-36428
> widened it to DECIMAL(16, 6), the cast drops the high bits, and an
> out-of-range seconds value can wrap into a valid one. For example:
> SELECT make_timestamp_ntz(2020, 1, 1, 0, 0, 4324.967296);
> -- 2020-01-01 00:00:30
> 4324967296 micros wraps to 30000000, exactly 30 seconds, so the query returns
> a timestamp instead of null (ANSI off) or DATETIME_FIELD_OUT_OF_BOUNDS (ANSI
> on), which is what an out-of-range value such as 61 seconds gives. The same
> holds for make_timestamp, for both the interpreted and the generated code,
> and in both modes. Values that wrap to a negative number already fail, but
> their ANSI error reports the wrapped seconds.
> Affects 3.2.0 and later.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
