Maxim Gekk created SPARK-31221: ---------------------------------- Summary: Rebase all dates/timestamps in conversion in Java types Key: SPARK-31221 URL: https://issues.apache.org/jira/browse/SPARK-31221 Project: Spark Issue Type: Bug Components: SQL Affects Versions: 3.0.0 Reporter: Maxim Gekk
Currently, the fromJavaDate(), toJavaDate(), toJavaTimestamp() and fromJavaTimestamp() methods of DateTimeUtils perform rebase only dates before Gregorian cutover date 1582-10-15 assuming that Gregorian calendar has the same behavior in Java 7 and Java 8 API. The assumption is incorrect, in particular, in getting zone offsets, for instance: {code:scala} scala> java.time.ZoneId.systemDefault res16: java.time.ZoneId = America/Los_Angeles scala> java.sql.Timestamp.valueOf("1883-11-10 00:00:00").getTimezoneOffset / 60.0 warning: there was one deprecation warning; re-run with -deprecation for details res17: Double = 8.0 scala> java.time.ZoneId.of("America/Los_Angeles").getRules.getOffset(java.time.LocalDateTime.parse("1883-11-10T00:00:00")) res18: java.time.ZoneOffset = -07:52:58 {code} Java 7 is not accurate, America/Los_Angeles changed time zone shift from {code} -7:52:58 {code} to {code} -8:00 {code} The ticket aims to perform rebasing for any dates/timestamps independently from calendar cutover date. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org