Github user MaxGekk commented on a diff in the pull request:
https://github.com/apache/spark/pull/23196#discussion_r238853314
--- Diff:
sql/hive/compatibility/src/test/scala/org/apache/spark/sql/hive/execution/HiveCompatibilitySuite.scala
---
@@ -49,8 +49,8 @@ class HiveCompatibilitySuite extends HiveQueryFileTest
with BeforeAndAfter {
override def beforeAll() {
super.beforeAll()
TestHive.setCacheTables(true)
- // Timezone is fixed to America/Los_Angeles for those timezone
sensitive tests (timestamp_*)
- TimeZone.setDefault(TimeZone.getTimeZone("America/Los_Angeles"))
+ // Timezone is fixed to GMT for those timezone sensitive tests
(timestamp_*)
--- End diff --
While porting on new parser/formatter, I faced to 2problems at least:
- The time zone from SQL config is not taken into account on parsing at
all. Basically used functions take default time zone from jvm settings. It
could be fixed by `TimeZone.setDefault` or using absolute values.
- Round trip in parsing a date to `DateType` and back to a date as a
string could give different string because `DateType` stores only days (as
`Int`) since epoch (in `UTC`). And such representation loses time zone offset.
So, exact matching is impossible due to lack of information. For example,
roundtrip converting for `TimestampType` works perfectly. This is the case for
the changes. Previously, it worked because the specified time zone is not used
at all (did not impact on number of days while converting a string to
`DateType`). With new parser/formatter, it becomes matter, and I have to change
time zone to `GMT` to eliminate the problem of loosing timezone offsets (it is
zero for `GMT`).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
