Re: Window function in Spark SQL

ontext Hive Server > should be running ? > > Regards, > Sourav > > > -- Forwarded message -- > From: <ross.cramb...@thomsonreuters.com> > Date: Fri, Dec 11, 2015 at 11:39 AM > Subject: Re: Window function in Spark SQL > To: sourav.mazumde.

Window function in Spark SQL

Hi, Spark SQL documentation says that it complies with Hive 1.2.1 APIs and supports Window functions. I'm using Spark 1.5.0. However, when I try to execute something like below I get an error val lol5 = sqlContext.sql("select ky, lead(ky, 5, 0) over (order by ky rows 5 following) from lolt")

Re: Window function in Spark SQL

Hey Sourav, Window functions require using a HiveContext rather than the default SQLContext. See here: http://spark.apache.org/docs/latest/sql-programming-guide.html#starting-point-sqlcontext HiveContext provides all the same functionality of SQLContext, as well as extra features like Window

Fwd: Window function in Spark SQL

that to create HIveContext Hive Server should be running ? Regards, Sourav -- Forwarded message -- From: <ross.cramb...@thomsonreuters.com> Date: Fri, Dec 11, 2015 at 11:39 AM Subject: Re: Window function in Spark SQL To: sourav.mazumde...@gmail.com I’m not familiar with that

Window function by Spark SQL

Hi, ALL How can I group by one column and order by another one, then select the first row for each group (which is just like window function doing) by SparkSQL？ Best Regards, Kevin.

Re: Window function by Spark SQL

Window functions are not supported yet, but there is a PR for it: https://github.com/apache/spark/pull/2953 On 12/5/14 12:22 PM, Dai, Kevin wrote: Hi, ALL How can I group by one column and order by another one, then select the first row for each group (which is just like window function