Regarding backticks: Right. You need backticks to quote the column name timestamp because timestamp is a reserved keyword in our parser.
On Tue, Feb 10, 2015 at 3:02 PM, Mohnish Kodnani <mohnish.kodn...@gmail.com> wrote: > actually i tried in spark shell , got same error and then for some reason > i tried to back tick the "timestamp" and it worked. > val result = sqlContext.sql("select toSeconds(`timestamp`) as t, > count(rid) as qps from blah group by toSeconds(`timestamp`),qi.clientName") > > so, it seems sql context is supporting UDF. > > > > On Tue, Feb 10, 2015 at 2:32 PM, Michael Armbrust <mich...@databricks.com> > wrote: > >> The simple SQL parser doesn't yet support UDFs. Try using a HiveContext. >> >> On Tue, Feb 10, 2015 at 1:44 PM, Mohnish Kodnani < >> mohnish.kodn...@gmail.com> wrote: >> >>> Hi, >>> I am trying a very simple registerFunction and it is giving me errors. >>> >>> I have a parquet file which I register as temp table. >>> Then I define a UDF. >>> >>> def toSeconds(timestamp: Long): Long = timestamp/100000 >>> >>> sqlContext.registerFunction("toSeconds", toSeconds _) >>> >>> val result = sqlContext.sql("select toSeconds(timestamp) from blah"); >>> I get the following error. >>> java.lang.RuntimeException: [1.18] failure: ``)'' expected but >>> `timestamp' found >>> >>> select toSeconds(timestamp) from blah >>> >>> My end goal is as follows: >>> We have log file with timestamps in microseconds and I would like to >>> group by entries with second level precision, so eventually I want to run >>> the query >>> select toSeconds(timestamp) as t, count(x) from table group by t,x >>> >>> >>> >>> >> >