[
https://issues.apache.org/jira/browse/SPARK-6385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14378689#comment-14378689
]
Michael Armbrust commented on SPARK-6385:
-----------------------------------------
[~bruun] I think all you need to do is add another S to the format spec.
{code}
scala> val ISO8601GMT: SimpleDateFormat = new SimpleDateFormat(
"yyyy-MM-dd'T'HH:mm:ss.SSSz" )
ISO8601GMT: java.text.SimpleDateFormat = java.text.SimpleDateFormat@8a9df61b
scala> ISO8601GMT.parse("2015-02-02T00:00:07.900GMT-00:00")
res0: java.util.Date = Sun Feb 01 16:00:07 PST 2015
scala> ISO8601GMT.parse("2015-02-02T00:00:07.9000GMT-00:00")
java.text.ParseException: Unparseable date: "2015-02-02T00:00:07.9000GMT-00:00"
at java.text.DateFormat.parse(DateFormat.java:357)
at .<init>(<console>:10)
at .<clinit>(<console>)
at .<init>(<console>:7)
at .<clinit>(<console>)
at $print(<console>)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:734)
at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:983)
at scala.tools.nsc.interpreter.IMain.loadAndRunReq$1(IMain.scala:573)
at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:604)
at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:568)
at scala.tools.nsc.interpreter.ILoop.reallyInterpret$1(ILoop.scala:756)
at
scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:801)
at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:713)
at scala.tools.nsc.interpreter.ILoop.processLine$1(ILoop.scala:577)
at scala.tools.nsc.interpreter.ILoop.innerLoop$1(ILoop.scala:584)
at scala.tools.nsc.interpreter.ILoop.loop(ILoop.scala:587)
at
scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:878)
at
scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:833)
at
scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:833)
at
scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:135)
at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:833)
at
scala.tools.nsc.MainGenericRunner.runTarget$1(MainGenericRunner.scala:83)
at scala.tools.nsc.MainGenericRunner.process(MainGenericRunner.scala:96)
at scala.tools.nsc.MainGenericRunner$.main(MainGenericRunner.scala:105)
at scala.tools.nsc.MainGenericRunner.main(MainGenericRunner.scala)
scala> val ISO8601GMT: SimpleDateFormat = new SimpleDateFormat(
"yyyy-MM-dd'T'HH:mm:ss.SSSSz" )
ISO8601GMT: java.text.SimpleDateFormat = java.text.SimpleDateFormat@c920c906
scala> ISO8601GMT.parse("2015-02-02T00:00:07.9000GMT-00:00")
res2: java.util.Date = Sun Feb 01 16:00:16 PST 2015
scala> ISO8601GMT.parse("2015-02-02T00:00:07.900GMT-00:00")
res3: java.util.Date = Sun Feb 01 16:00:07 PST 2015
{code}
> ISO 8601 timestamp parsing does not support arbitrary precision second
> fractions
> --------------------------------------------------------------------------------
>
> Key: SPARK-6385
> URL: https://issues.apache.org/jira/browse/SPARK-6385
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 1.2.1
> Reporter: Nick Bruun
> Priority: Minor
>
> The ISO 8601 timestamp parsing implemented as a resolution to SPARK-4149 does
> not support arbitrary precision fractions of seconds, only millisecond
> precision. Parsing {{2015-02-02T00:00:07.900GMT-00:00}} will succeed, while
> {{2015-02-02T00:00:07.9000GMT-00:00}} will fail.
> The issue is caused by the fixed precision of the parsed format in
> [DataTypeConversions.scala#L66|https://github.com/apache/spark/blob/84acd08e0886aa23195f35837c15c09aa7804aff/sql/catalyst/src/main/scala/org/apache/spark/sql/types/DataTypeConversions.scala#L66].
> I'm willing to implement a fix, but pointers on the direction would be
> appreciated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
