Cheng thanks, thanks to you I found out that the problem as you guessed was a precision one.
2012-10-08 16:10:36 instead of 2012-10-08 16:10:36.0 Thanks again. Alessandro > On Nov 23, 2014, at 11:10 PM, Cheng, Hao [via Apache Spark User List] > <ml-node+s1001560n19613...@n3.nabble.com> wrote: > > Can you try query like “SELECT timestamp, CAST(timestamp as string) FROM logs > LIMIT 5”, I guess you probably ran into the timestamp precision or the > timezone shifting problem. > > > > (And it’s not mandatory, but you’d better change the field name from > “timestamp” to something else, as “timestamp” is the keyword of data type in > Hive/Spark SQL.) > > <> > From: Alessandro Panebianco [mailto:[hidden email] > <x-msg://2/user/SendEmail.jtp?type=node&node=19613&i=0>] > Sent: Monday, November 24, 2014 11:12 AM > To: Wang, Daoyuan > Cc: [hidden email] <x-msg://2/user/SendEmail.jtp?type=node&node=19613&i=1> > Subject: Re: SparkSQL Timestamp query failure > > > > Hey Daoyuan, > > > > following your suggestion I obtain the same result as when I do: > > > > where l.timestamp = '2012-10-08 16:10:36.0’ > > > > what happens using either your suggestion or simply using single quotes as I > just typed in the example before is that the query does not fail but it > doesn’t return anything either as it should. > > > > If I do a simple : > > > > SELECT timestamp FROM Logs limit 5").collect.foreach(println) > > > > I get: > > > > [2012-10-08 16:10:36.0] > > [2012-10-08 16:10:36.0] > > [2012-10-08 16:10:36.0] > > [2012-10-08 16:10:41.0] > > [2012-10-08 16:10:41.0] > > > > that is why I am sure that putting one of those timestamps should not return > an empty arrray. > > > > Id really love to find a solution to this problem. Since Spark supports > Timestamp it should provide simple comparison actions with them in my opinion. > > > > Any other help would be greatly appreciated. > > > > Alessandro > > > > > > > > > > On Nov 23, 2014, at 8:10 PM, Wang, Daoyuan <[hidden email] > <x-msg://2/user/SendEmail.jtp?type=node&node=19613&i=2>> wrote: > > > > Hi, > > I think you can try > cast(l.timestamp as string)='2012-10-08 16:10:36.0' > > Thanks, > Daoyuan > > -----Original Message----- > From: whitebread [[hidden email] > <x-msg://2/user/SendEmail.jtp?type=node&node=19613&i=3>] > Sent: Sunday, November 23, 2014 12:11 AM > To: [hidden email] <x-msg://2/user/SendEmail.jtp?type=node&node=19613&i=4> > Subject: Re: SparkSQL Timestamp query failure > > Thanks for your answer Akhil, > > I have already tried that and the query actually doesn't fail but it doesn't > return anything either as it should. > Using single quotes I think it reads it as a string and not as a timestamp. > > I don't know how to solve this. Any other hint by any chance? > > Thanks, > > Alessandro > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/SparkSQL-Timestamp-query-failure-tp19502p19554.html > > <http://apache-spark-user-list.1001560.n3.nabble.com/SparkSQL-Timestamp-query-failure-tp19502p19554.html> > Sent from the Apache Spark User List mailing list archive at Nabble.com > <http://nabble.com/>. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [hidden email] > <x-msg://2/user/SendEmail.jtp?type=node&node=19613&i=5> For additional > commands, e-mail: [hidden email] > <x-msg://2/user/SendEmail.jtp?type=node&node=19613&i=6> > > > > > If you reply to this email, your message will be added to the discussion > below: > http://apache-spark-user-list.1001560.n3.nabble.com/SparkSQL-Timestamp-query-failure-tp19502p19613.html > > <http://apache-spark-user-list.1001560.n3.nabble.com/SparkSQL-Timestamp-query-failure-tp19502p19613.html> > To unsubscribe from SparkSQL Timestamp query failure, click here > <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=19502&code=YWxlLnBhbmViaWFuY29AbWUuY29tfDE5NTAyfC00MjA1ODk4MTE=>. > NAML > <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/SparkSQL-Timestamp-query-failure-tp19502p19616.html Sent from the Apache Spark User List mailing list archive at Nabble.com.
