Maxim Gekk created SPARK-26740:
----------------------------------
Summary: Statistics for date and timestamp columns depend on
system time zone
Key: SPARK-26740
URL: https://issues.apache.org/jira/browse/SPARK-26740
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 2.4.0
Reporter: Maxim Gekk
While saving statistics for timestamp/date columns, default time zone is used
in conversion of internal type (microseconds or days since epoch) to textual
representation. The textual representation doesn't contain time zone. So, when
it is converted back to internal types (Long for TimestampType or DateType),
the Timestamp.valueOf and Date.valueOf are used in conversions. The methods use
current system time zone.
If system time zone is different while saving and retrieving statistics for
timestamp/date columns, restored microseconds/days since epoch will be
different.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
