Re: GraphFrames 0.2.0 released

Hi Tim. Could you share link to the release docs as well? Thanks, Shagun https://twitter.com/shagunsodhani On Tue, Aug 16, 2016 at 10:02 PM, Tim Hunter wrote: > Hello all, > I have released version 0.2.0 of the GraphFrames package. Apart from a few > bug fixes, it is

Re: Lead operator not working as aggregation operator

I was referring to this jira issue : https://issues.apache.org/jira/browse/TAJO-919 On Mon, Nov 2, 2015 at 4:03 PM, Shagun Sodhani <sshagunsodh...@gmail.com> wrote: > Hi! I was trying out window functions in SparkSql (using hive context) > and I noticed that while t

Lead operator not working as aggregation operator

Hi! I was trying out window functions in SparkSql (using hive context) and I noticed that while this mentions that *lead* is implemented as an aggregate operator, it seems not to be the case. I am using the

Re: Lead operator not working as aggregation operator

fine: > > *select lead(max(expenses)) over (order by customerId) from tbl group by > customerId* > > HTH > > Met vriendelijke groet/Kind regards, > > Herman van Hövell tot Westerflier > > QuestTec B.V. > Torenwacht 98 > 2353 DC Leiderdorp > hvanhov...@questtec.

Re: Exception when using some aggregate operators

I tried adding the aggregate functions in the registry and they work, other than mean, for which Ted has forwarded some code changes. I will try out those changes and update the status here. On Wed, Oct 28, 2015 at 9:03 AM, Shagun Sodhani <sshagunsodh...@gmail.com> wrote: > Yup avg w

Re: Exception when using some aggregate operators

ast_value"), > expression[Max]("max"), > +expression[Average]("mean"), > expression[Min]("min"), > expression[Stddev]("stddev"), > expression[StddevPop]("stddev_pop"), > > FYI > > On Wed, Oct 28

Re: Exception when using some aggregate operators

Also are the other aggregate functions to be treated as bugs or not? On Wed, Oct 28, 2015 at 4:08 PM, Shagun Sodhani <sshagunsodh...@gmail.com> wrote: > Wouldnt it be: > > +expression[Max]("avg"), > > On Wed, Oct 28, 2015 at 4:06 PM, Ted Yu <yuzh

Re: Exception when using some aggregate operators

quot;mean". Similarly, a distinct count is supposed to be written as > "count(distinct col)", not "countDistinct(col)". > > We can, however, make "mean" an alias for "avg" to improve compatibility > between DataFrame and SQL. > > > On W

Re: Exception when using some aggregate operators

Ohh great! Thanks for the clarification. On Wed, Oct 28, 2015 at 4:21 PM, Reynold Xin <r...@databricks.com> wrote: > No those are just functions for the DataFrame programming API. > > On Wed, Oct 28, 2015 at 11:49 AM, Shagun Sodhani <sshagunsodh...@gmail.com > > wro

Exception when using some aggregate operators

Hi! I was trying out some aggregate functions in SparkSql and I noticed that certain aggregate operators are not working. This includes: approxCountDistinct countDistinct mean sumDistinct For example using countDistinct results in an error saying *Exception in thread "main"

Re: Exception when using some aggregate operators

sql(""" > CREATE TEMPORARY TABLE partitionedParquet > USING org.apache.spark.sql.parquet > OPTIONS ( > path '/tmp/partitioned' > )""") > sqlContext.sql("""select avg(a) from partitionedParquet""").sho

Re: Exception when using some aggregate operators

Oops seems I made a mistake. The error message is : Exception in thread "main" org.apache.spark.sql.AnalysisException: undefined function countDistinct On 27 Oct 2015 15:49, "Shagun Sodhani" <sshagunsodh...@gmail.com> wrote: > Hi! I was trying out some aggregate

Adding support for truncate operator

Hi! I noticed that SparkSQL does not support truncate operator as of now. Can we add it? I am willing to send over a PR for it

Re: Adding support for truncate operator

gt; On Sun, Oct 25, 2015 at 9:01 AM, Shagun Sodhani <sshagunsodh...@gmail.com> > wrote: > >> Hi! I noticed that SparkSQL does not support truncate operator as of now. >> Can we add it? I am willing to send over a PR for it >> > >

Re: Exception when using cosh

rk/sql/catalyst/analysis/FunctionRegistry.scala > > Do you mind submitting a pull request to fix this? Should be an one line > change. I filed a ticket to track this: > https://issues.apache.org/jira/browse/SPARK-11233 > > > > > On Wed, Oct 21, 2015 at 2:30 AM, Shagun

Re: Exception when using cosh

@Reynold submitted the PR: https://github.com/apache/spark/pull/9199 On Wed, Oct 21, 2015 at 11:01 PM, Shagun Sodhani <sshagunsodh...@gmail.com> wrote: > Sure! Would do that. > > Thanks a lot > > On Wed, Oct 21, 2015 at 10:59 PM, Reynold Xin <r...@databricks.com>

Exception when using cosh

Hi! I was trying out different arithmetic functions in SparkSql. I noticed a weird thing. While *sinh* and *tanh* functions are working, using *cosh* results in an error saying: *Exception in thread "main" org.apache.spark.sql.AnalysisException: undefined function cosh;* The documentation says