sepuri sai krishna created SPARK-60910:
------------------------------------------
Summary: datediff silently wraps around on integer overflow
Key: SPARK-60910
URL: https://issues.apache.org/jira/browse/SPARK-60910
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 4.0.3, 5.0.0, 4.4.0
Reporter: sepuri sai krishna
{{datediff}} / {{date_diff}} subtract the two day counts as an {{INT}}, so a
difference that does
not fit in an {{INT}} silently wraps around, with ANSI mode on or off and in
both interpreted and
generated code. On 4.0.3 and master, on values read from a table:
{code:sql}
CREATE TABLE dd USING parquet AS
SELECT date_from_unix_date(2147483647) AS e, date_from_unix_date(-1) AS s,
date_from_unix_date(-2147483648) AS mn;
SELECT datediff(e, s) FROM dd; -- -2147483648, should be 2147483648
SELECT datediff(e, mn) FROM dd; -- -1, should be 4294967295
{code}
On the same values, the related operations fail with a raw
{{java.lang.ArithmeticException}}
instead of an error class:
{code:sql}
SELECT e - s FROM dd; -- java.lang.ArithmeticException:
integer overflow
SELECT timestampdiff(DAY, e, s) FROM dd; -- java.lang.ArithmeticException: long
overflow
SELECT months_between(e, s) FROM dd; -- java.lang.ArithmeticException: long
overflow
{code}
{{DateDiff.nullSafeEval}} is {{end - start}} on {{Int}} (same in codegen). This
is the same class
of bug as SPARK-59859 ({{date_add}} / {{date_sub}}), and was raised in the
review of
https://github.com/apache/spark/pull/59135.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
