[
https://issues.apache.org/jira/browse/SPARK-40791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17619681#comment-17619681
]
Yang Jie commented on SPARK-40791:
----------------------------------
Yes, when run test in `datetime-formatting.sql` with Java 19, there are related
test failure/behavior change, the test failure reason is as described above,
Java 19 has corrected its own behavior, and the fix will also be carried by the
next LTS. Perhaps it is appropriate for us to correct this behavior in Spark
4.0?
I can try to make the behavior of this issue to keep consistent with current
one if needed, although Java 19 is just a STS .
> The semantics of `F` in `DateTimeFormatter` have changed
> --------------------------------------------------------
>
> Key: SPARK-40791
> URL: https://issues.apache.org/jira/browse/SPARK-40791
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Affects Versions: 3.4.0
> Reporter: Yang Jie
> Priority: Major
>
> {code:java}
> val createSql =
> """
> |create temporary view v as select col from values
> | (timestamp '1582-06-01 11:33:33.123UTC+080000'),
> | (timestamp '1970-01-01 00:00:00.000Europe/Paris'),
> | (timestamp '1970-12-31 23:59:59.999Asia/Srednekolymsk'),
> | (timestamp '1996-04-01 00:33:33.123Australia/Darwin'),
> | (timestamp '2018-11-17 13:33:33.123Z'),
> | (timestamp '2020-01-01 01:33:33.123Asia/Shanghai'),
> | (timestamp '2100-01-01 01:33:33.123America/Los_Angeles') t(col)
> | """.stripMargin
> sql(createSql)
> withSQLConf(SQLConf.WHOLESTAGE_CODEGEN_ENABLED.key -> false.toString) {
> val rows = sql("select col, date_format(col, 'F') from v").collect()
> // scalastyle:off
> rows.foreach(println)
> } {code}
>
> Before Java 19, the result is
>
> {code:java}
> [1582-05-31 19:40:35.123,3]
> [1969-12-31 15:00:00.0,3]
> [1970-12-31 04:59:59.999,3]
> [1996-03-31 07:03:33.123,3]
> [2018-11-17 05:33:33.123,3]
> [2019-12-31 09:33:33.123,3]
> [2100-01-01 01:33:33.123,1] {code}
> Java 19
>
> {code:java}
> [1582-05-31 19:40:35.123,5]
> [1969-12-31 15:00:00.0,5]
> [1970-12-31 04:59:59.999,5]
> [1996-03-31 07:03:33.123,5]
> [2018-11-17 05:33:33.123,3]
> [2019-12-31 09:33:33.123,5]
> [2100-01-01 01:33:33.123,1] {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
