[ https://issues.apache.org/jira/browse/SPARK-30752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17031774#comment-17031774 ]
Maxim Gekk commented on SPARK-30752: ------------------------------------ [~dongjoon] FYI, 2.4 has the bug. > Wrong result of to_utc_timestamp() on daylight saving day > --------------------------------------------------------- > > Key: SPARK-30752 > URL: https://issues.apache.org/jira/browse/SPARK-30752 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 2.4.4, 3.0.0 > Reporter: Maxim Gekk > Priority: Major > > The to_utc_timestamp() function returns wrong result when: > * JVM system time zone is PST > * the session local time zone is UTC > * fromZone is Asia/Hong_Kong > for the local date '2019-11-03T12:00:00', the result must be > '2019-11-03T04:00:00' > {code} > scala> import java.util.TimeZone > import java.util.TimeZone > scala> import org.apache.spark.sql.catalyst.util.DateTimeUtils._ > import org.apache.spark.sql.catalyst.util.DateTimeUtils._ > scala> import org.apache.spark.sql.functions._ > import org.apache.spark.sql.functions._ > scala> TimeZone.setDefault(getTimeZone("PST")) > scala> spark.conf.set("spark.sql.session.timeZone", "UTC") > scala> val df = Seq("2019-11-03T12:00:00").toDF("localTs") > df: org.apache.spark.sql.DataFrame = [localTs: string] > scala> df.select(to_utc_timestamp(col("localTs"), "Asia/Hong_Kong")).show > +-----------------------------------------+ > |to_utc_timestamp(localTs, Asia/Hong_Kong)| > +-----------------------------------------+ > | 2019-11-03 03:00:00| > +-----------------------------------------+ > {code} > > See > https://www.worldtimebuddy.com/?qm=1&lid=8,1819729,100&h=8&date=2019-11-2&sln=21-22 -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org