MaxGekk opened a new pull request #27457: [SPARK-30730][SQL] Fix wrong assumption of `DateTimeUtils.converTz` URL: https://github.com/apache/spark/pull/27457 ### What changes were proposed in this pull request? In the PR, I propose to use the session local time zone defined by the SQL config `spark.sql.session.timeZone` instead of system JVM time zone in `DateTimeUtils.converTz`. In the case when those time zones are different, `convertTz` assumes wrongly that its input timestamp was parsed from local timestamp string using JVM time zone. Such assumption was correct in the past when timestamp strings were parsed by `java.sql.Timestamp.valueOf`. Currently, timestamp strings are converted to timestamp value via `CAST` which uses the session local time zone. ### Why are the changes needed? This fixes wrong behavior of the `to_utc_timestamp` and `from_utc_timestamp` function in the case when system JVM time zone is different from session time zone. ### Does this PR introduce any user-facing change? Yes, when session local time zone is different from system JVM time zone. ### How was this patch tested? Add new tests to `DateFunctionsSuite` for various JVM time zone, session time zone and to/from time zones.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
