that explains a lot...
Is there a list of reserved keywords ?
On Wed, Feb 11, 2015 at 7:56 AM, Yin Huai <yh...@databricks.com> wrote:
> Regarding backticks: Right. You need backticks to quote the column name
> timestamp because timestamp is a reserved keyword in our parser.
>
> On Tue, Feb 10, 2015 at 3:02 PM, Mohnish Kodnani <
> mohnish.kodn...@gmail.com> wrote:
>
>> actually i tried in spark shell , got same error and then for some reason
>> i tried to back tick the "timestamp" and it worked.
>> val result = sqlContext.sql("select toSeconds(`timestamp`) as t,
>> count(rid) as qps from blah group by toSeconds(`timestamp`),qi.clientName")
>>
>> so, it seems sql context is supporting UDF.
>>
>>
>>
>> On Tue, Feb 10, 2015 at 2:32 PM, Michael Armbrust <mich...@databricks.com
>> > wrote:
>>
>>> The simple SQL parser doesn't yet support UDFs. Try using a HiveContext.
>>>
>>> On Tue, Feb 10, 2015 at 1:44 PM, Mohnish Kodnani <
>>> mohnish.kodn...@gmail.com> wrote:
>>>
>>>> Hi,
>>>> I am trying a very simple registerFunction and it is giving me errors.
>>>>
>>>> I have a parquet file which I register as temp table.
>>>> Then I define a UDF.
>>>>
>>>> def toSeconds(timestamp: Long): Long = timestamp/100000
>>>>
>>>> sqlContext.registerFunction("toSeconds", toSeconds _)
>>>>
>>>> val result = sqlContext.sql("select toSeconds(timestamp) from blah");
>>>> I get the following error.
>>>> java.lang.RuntimeException: [1.18] failure: ``)'' expected but
>>>> `timestamp' found
>>>>
>>>> select toSeconds(timestamp) from blah
>>>>
>>>> My end goal is as follows:
>>>> We have log file with timestamps in microseconds and I would like to
>>>> group by entries with second level precision, so eventually I want to run
>>>> the query
>>>> select toSeconds(timestamp) as t, count(x) from table group by t,x
>>>>
>>>>
>>>>
>>>>
>>>
>>
>
