Hello Friends:
Here's sample output from a SparkSQL query that works, just so you can
see the
underlying data structure; followed by one that fails.
>>> # Just you you can see the DataFrame structure ...
>>>
>>> resultsRDD = sqlCtx.sql("SELECT * FROM rides WHERE
trip_time_in_secs = 3780")
>>> resultsRDD.collect() # WORKS.
[Row(pickup_datetime=datetime.datetime(2013, 10, 4, 8, 0),
dropoff_datetime=datetime.datetime(2013, 10, 4, 9, 3),
trip_time_in_secs=3780,
trip_distance=17.100000381469727),
Row(pickup_datetime=datetime.datetime(2013, 10, 18, 8, 0),
dropoff_datetime=datetime.datetime(2013, 10, 18, 9, 3),
trip_time_in_secs=3780,
trip_distance=17.920000076293945), ... )
But the following SQL experiences the exception shown below.
>>> resultsRDD = sqlCtx.sql("SELECT * FROM rides WHERE pickup_datetime
> datetime.datetime(2013,12,1,0,0,0)")
>>> resultsRDD.collect() # FAILS.
/ py4j.protocol.Py4JJavaError: An error occurred while calling o53.sql.//
// java.lang.RuntimeException: [1.62] failure: ``union'' expected but
`(' found/
It's the first time I'm trying this and seemingly doing it incorrectly.
Can anyone show me how to correct this?
Thank you! =:)
nmv
--
PRISMALYTICS Sincerely yours,
Team PRISMALYTICS
------------------------------------------------------------------------
PRISMALYTICS, LLC. <http://www.prismalytics.com/> | www.prismalytics.com
<http://www.prismalytics.com/>
P: 212.882.1276 <tel:212.882.1276> | subscripti...@prismalytics.io
<mailto:subscripti...@prismalytics.io>
Follow Us: https://www.LinkedIn.com/company/prismalytics
<https://www.linkedin.com/company/prismalytics>
Prismalytics, LLC. <http://www.prismalytics.com/>
data analytics to literally count on
