Gengliang Wang created SPARK-46396:
--------------------------------------
Summary: LegacyFastTimestampFormatter.parseOptional should not
throw exception
Key: SPARK-46396
URL: https://issues.apache.org/jira/browse/SPARK-46396
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 3.5.0
Reporter: Gengliang Wang
Assignee: Gengliang Wang
When setting spark.sql.legacy.timeParserPolicy=LEGACY, Spark will use the
LegacyFastTimestampFormatter to infer potential timestamp columns. The
inference shouldn't throw exception.
However, when the input is 23012150952, there is exception:
```
For input string: "23012150952"
java.lang.NumberFormatException: For input string: "23012150952"
at
java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:67)
at java.base/java.lang.Integer.parseInt(Integer.java:668)
at java.base/java.lang.Integer.parseInt(Integer.java:786)
at
org.apache.commons.lang3.time.FastDateParser$NumberStrategy.parse(FastDateParser.java:304)
at org.apache.commons.lang3.time.FastDateParser.parse(FastDateParser.java:1045)
at org.apache.commons.lang3.time.FastDateFormat.parse(FastDateFormat.java:651)
at
org.apache.spark.sql.catalyst.util.LegacyFastTimestampFormatter.parseOptional(TimestampFormatter.scala:418)
```
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
