[
https://issues.apache.org/jira/browse/SPARK-31892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-31892:
------------------------------------
Assignee: (was: Apache Spark)
> Disable week-based date filed for parsing
> -----------------------------------------
>
> Key: SPARK-31892
> URL: https://issues.apache.org/jira/browse/SPARK-31892
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Affects Versions: 3.0.0, 3.1.0
> Reporter: Kent Yao
> Priority: Major
>
> It's an un-fixable behavior change to fill the gap between SimpleDateFormat
> and DateTimeFormater and backward-compatibility for different JDKs.
> A lot of effort has been made to prove this at
> https://github.com/apache/spark/pull/28674
> The existing behavior itself in 2.4 is confusing, e.g.
> {code:sql}
> spark-sql> select to_timestamp('1', 'w');
> 1969-12-28 00:00:00
> spark-sql> select to_timestamp('1', 'u');
> 1970-01-05 00:00:00
> {code}
> the 'u' here seems not to go to the Monday of the first week in week-based
> form or the first day of the year in non-week-based form but go to the Monday
> of the second week in week-based form.
> {code:sql}
> spark-sql> select to_timestamp('2020 2020', 'YYYY yyyy');
> 2020-01-01 00:00:00
> spark-sql> select to_timestamp('2020 2020', 'yyyy YYYY');
> 2019-12-29 00:00:00
> spark-sql> select to_timestamp('2020 2020 1', 'YYYY yyyy w');
> NULL
> spark-sql> select to_timestamp('2020 2020 1', 'yyyy YYYY w');
> 2019-12-29 00:00:00
> {code}
> I think we don't need to introduce all the weird behavior from Java.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
