Re: spark sql registerFunction with 1.2.1

2015-02-11 Thread Yin Huai
Regarding backticks: Right. You need backticks to quote the column name timestamp because timestamp is a reserved keyword in our parser. On Tue, Feb 10, 2015 at 3:02 PM, Mohnish Kodnani mohnish.kodn...@gmail.com wrote: actually i tried in spark shell , got same error and then for some reason i

Re: spark sql registerFunction with 1.2.1

2015-02-11 Thread Mohnish Kodnani
that explains a lot... Is there a list of reserved keywords ? On Wed, Feb 11, 2015 at 7:56 AM, Yin Huai yh...@databricks.com wrote: Regarding backticks: Right. You need backticks to quote the column name timestamp because timestamp is a reserved keyword in our parser. On Tue, Feb 10, 2015

spark sql registerFunction with 1.2.1

2015-02-10 Thread Mohnish Kodnani
Hi, I am trying a very simple registerFunction and it is giving me errors. I have a parquet file which I register as temp table. Then I define a UDF. def toSeconds(timestamp: Long): Long = timestamp/10 sqlContext.registerFunction(toSeconds, toSeconds _) val result = sqlContext.sql(select

Re: spark sql registerFunction with 1.2.1

2015-02-10 Thread Michael Armbrust
The simple SQL parser doesn't yet support UDFs. Try using a HiveContext. On Tue, Feb 10, 2015 at 1:44 PM, Mohnish Kodnani mohnish.kodn...@gmail.com wrote: Hi, I am trying a very simple registerFunction and it is giving me errors. I have a parquet file which I register as temp table. Then I