Re: percentil UDAF in spark 1.2.0

...@clearstorydata.com wrote: Already fixed: https://github.com/apache/spark/pull/2802 On Thu, Feb 19, 2015 at 3:17 PM, Mohnish Kodnani mohnish.kodn...@gmail.com wrote: Hi, I am trying to use percentile and getting the following error. I am using spark 1.2.0. Does UDAF percentile exist

percentil UDAF in spark 1.2.0

Hi, I am trying to use percentile and getting the following error. I am using spark 1.2.0. Does UDAF percentile exist in that code line and do i have to do something to get this to work. java.util.NoSuchElementException: key not found: percentile at

Re: exception with json4s render

not using it. I've downgraded to 3.2.10 and it seems to work. I searched through the spark repo and it looks like it's got 3.2.10 in a pom. I don't know the first thing about how dependencies are resolved but I'm guessing it's related? On Wed Feb 11 2015 at 11:20:42 AM Mohnish Kodnani

Re: exception with json4s render

and it looks like it's got 3.2.10 in a pom. I don't know the first thing about how dependencies are resolved but I'm guessing it's related? On Wed Feb 11 2015 at 11:20:42 AM Mohnish Kodnani mohnish.kodn...@gmail.com wrote: I was getting similar error after I upgraded to spark 1.2.1 from 1.1.1

Re: spark sql registerFunction with 1.2.1

at 3:02 PM, Mohnish Kodnani mohnish.kodn...@gmail.com wrote: actually i tried in spark shell , got same error and then for some reason i tried to back tick the timestamp and it worked. val result = sqlContext.sql(select toSeconds(`timestamp`) as t, count(rid) as qps from blah group

Re: exception with json4s render

I was getting similar error after I upgraded to spark 1.2.1 from 1.1.1 Are you by any chance using json4s 3.2.11. I downgraded to 3.2.10 and that seemed to have worked. But I didnt try to spend much time debugging the issue than that. On Wed, Feb 11, 2015 at 11:13 AM, Jonathan Haddad

spark sql registerFunction with 1.2.1

Hi, I am trying a very simple registerFunction and it is giving me errors. I have a parquet file which I register as temp table. Then I define a UDF. def toSeconds(timestamp: Long): Long = timestamp/10 sqlContext.registerFunction(toSeconds, toSeconds _) val result = sqlContext.sql(select

Spark SQL group by

Hi, i am trying to issue a sql query against a parquet file and am getting errors and would like some help to figure out what is going on. The sql : select timestamp, count(rid), qi.clientname from records where timestamp 0 group by qi.clientname I am getting the following error:

Re: Spark SQL group by

Doh :) Thanks.. seems like brain freeze. On Fri, Feb 6, 2015 at 3:22 PM, Michael Armbrust mich...@databricks.com wrote: You can't use columns (timestamp) that aren't in the GROUP BY clause. Spark 1.2+ give you a better error message for this case. On Fri, Feb 6, 2015 at 3:12 PM, Mohnish

Load multiple parquet file as single RDD

Hi, I am trying to play around with Spark and Spark SQL. I have logs being stored in HDFS on a 10 minute window. Each 10 minute window could have as many as 10 files with random names of 2GB each. Now, I want to run some analysis on these files. These files are parquet files. I am trying to run