[ https://issues.apache.org/jira/browse/SPARK-31150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Wenchen Fan reassigned SPARK-31150: ----------------------------------- Assignee: Kent Yao > Parsing seconds fraction with variable length for timestamp > ----------------------------------------------------------- > > Key: SPARK-31150 > URL: https://issues.apache.org/jira/browse/SPARK-31150 > Project: Spark > Issue Type: Improvement > Components: SQL > Affects Versions: 3.1.0 > Reporter: Kent Yao > Assignee: Kent Yao > Priority: Major > > This JIRA is to support parsing timestamp values with variable length second > fraction parts. > e.g. 'yyyy-MM-dd HH:mm:ss.SSSSSS[zzz]' can parse timestamp with 0~6 > digit-length second fraction but fail >=7 > {code:java} > select to_timestamp(v, 'yyyy-MM-dd HH:mm:ss.SSSSSS[zzz]') from values > ('2019-10-06 10:11:12.'), > ('2019-10-06 10:11:12.0'), > ('2019-10-06 10:11:12.1'), > ('2019-10-06 10:11:12.12'), > ('2019-10-06 10:11:12.123UTC'), > ('2019-10-06 10:11:12.1234'), > ('2019-10-06 10:11:12.12345CST'), > ('2019-10-06 10:11:12.123456PST') t(v) > 2019-10-06 03:11:12.123 > 2019-10-06 08:11:12.12345 > 2019-10-06 10:11:12 > 2019-10-06 10:11:12 > 2019-10-06 10:11:12.1 > 2019-10-06 10:11:12.12 > 2019-10-06 10:11:12.1234 > 2019-10-06 10:11:12.123456 > select to_timestamp('2019-10-06 10:11:12.1234567PST', 'yyyy-MM-dd > HH:mm:ss.SSSSSS[zzz]') > NULL > {code} > Since 3.0, we use java 8 time API to parse and format timestamp values. when > we create the DateTimeFormatter, we use appendPattern to create the build > first, where the 'S..S' part will be parsed to a fixed-length(= > 'S..S'.length). This fits the formatting part but too strict for the parsing > part because the trailing zeros are very likely to be truncated. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org