[ https://issues.apache.org/jira/browse/SPARK-9794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14680688#comment-14680688 ]
Josh Rosen commented on SPARK-9794: ----------------------------------- The same code exists in 1.4.0 and 1.4.1: https://github.com/apache/spark/blob/v1.4.1/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateUtils.scala#L86 It's also present in 1.3.0 / 1.3.1: https://github.com/apache/spark/blob/v1.3.1/sql/catalyst/src/main/scala/org/apache/spark/sql/types/DataTypeConversions.scala#L66 And in 1.2.x: https://github.com/apache/spark/blob/v1.2.2/sql/core/src/main/scala/org/apache/spark/sql/types/util/DataTypeConversions.scala#L158 Here's the pull request that originally added that line: https://github.com/apache/spark/pull/3012 > ISO DateTime parser is too strict > --------------------------------- > > Key: SPARK-9794 > URL: https://issues.apache.org/jira/browse/SPARK-9794 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 1.2.2, 1.3.1, 1.4.1, 1.5.0 > Reporter: Alex Angelini > > The DateTime parser requires 3 millisecond digits, but that is not part of > the official ISO8601 spec. > https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala#L132 > https://en.wikipedia.org/wiki/ISO_8601 > This results in the following exception when trying to parse datetime columns > {code} > java.text.ParseException: Unparseable date: "0001-01-01T00:00:00GMT-00:00" > {code} > [~joshrosen] [~rxin] -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org