Re: GraphFrames 0.2.0 released

Hi Tim. Could you share link to the release docs as well? Thanks, Shagun https://twitter.com/shagunsodhani On Tue, Aug 16, 2016 at 10:02 PM, Tim Hunter wrote: > Hello all, > I have released version 0.2.0 of the GraphFrames package. Apart from a few > bug fixes, it is the first release publishe

Re: Lead operator not working as aggregation operator

> fine: > > *select lead(max(expenses)) over (order by customerId) from tbl group by > customerId* > > HTH > > Met vriendelijke groet/Kind regards, > > Herman van Hövell tot Westerflier > > QuestTec B.V. > Torenwacht 98 > 2353 DC Leiderdorp > hvanhov...@ques

Re: Lead operator not working as aggregation operator

I was referring to this jira issue : https://issues.apache.org/jira/browse/TAJO-919 On Mon, Nov 2, 2015 at 4:03 PM, Shagun Sodhani wrote: > Hi! I was trying out window functions in SparkSql (using hive context) > and I noticed that while this > <https://issues.apache.org/jira/bro

Lead operator not working as aggregation operator

Hi! I was trying out window functions in SparkSql (using hive context) and I noticed that while this mentions that *lead* is implemented as an aggregate operator, it seems not to be the case. I am using the followin

Re: Exception when using some aggregate operators

Ohh great! Thanks for the clarification. On Wed, Oct 28, 2015 at 4:21 PM, Reynold Xin wrote: > No those are just functions for the DataFrame programming API. > > On Wed, Oct 28, 2015 at 11:49 AM, Shagun Sodhani > wrote: > >> @Reynold I seem to be missing something. Aren&#

Re: Exception when using some aggregate operators

;. Similarly, a distinct count is supposed to be written as > "count(distinct col)", not "countDistinct(col)". > > We can, however, make "mean" an alias for "avg" to improve compatibility > between DataFrame and SQL. > > > On Wed, Oct 28, 2015 a

Re: Exception when using some aggregate operators

expression[Max]("max"), > +expression[Average]("mean"), > expression[Min]("min"), > expression[Stddev]("stddev"), > expression[StddevPop]("stddev_pop"), > > FYI > > On Wed, Oct 28, 2015 at 2:07 AM, Shagu

Re: Exception when using some aggregate operators

Also are the other aggregate functions to be treated as bugs or not? On Wed, Oct 28, 2015 at 4:08 PM, Shagun Sodhani wrote: > Wouldnt it be: > > +expression[Max]("avg"), > > On Wed, Oct 28, 2015 at 4:06 PM, Ted Yu wrote: > >> Since there is already A

Re: Exception when using some aggregate operators

I tried adding the aggregate functions in the registry and they work, other than mean, for which Ted has forwarded some code changes. I will try out those changes and update the status here. On Wed, Oct 28, 2015 at 9:03 AM, Shagun Sodhani wrote: > Yup avg works good. So we have altern

Re: Exception when using some aggregate operators

uot;"" > CREATE TEMPORARY TABLE partitionedParquet > USING org.apache.spark.sql.parquet > OPTIONS ( > path '/tmp/partitioned' > )""") > sqlContext.sql("""select avg(a) from partitionedParquet""").show() > > Cheers

Re: Exception when using some aggregate operators

2015 at 8:02 PM, Shagun Sodhani wrote: > Will try in a while when I get back. I assume this applies to all > functions other than mean. Also countDistinct is defined along with all > other SQL functions. So I don't get "distinct is not part of function name" > part. >

Re: Exception when using some aggregate operators

> Try > > count(distinct columnane) > > In SQL distinct is not part of the function name. > > On Tuesday, October 27, 2015, Shagun Sodhani > wrote: > >> Oops seems I made a mistake. The error message is : Exception in thread >> "main" org.apache.spark.sql.A

Re: Exception when using some aggregate operators

Oops seems I made a mistake. The error message is : Exception in thread "main" org.apache.spark.sql.AnalysisException: undefined function countDistinct On 27 Oct 2015 15:49, "Shagun Sodhani" wrote: > Hi! I was trying out some aggregate functions in SparkSql and I

Exception when using some aggregate operators

Hi! I was trying out some aggregate functions in SparkSql and I noticed that certain aggregate operators are not working. This includes: approxCountDistinct countDistinct mean sumDistinct For example using countDistinct results in an error saying *Exception in thread "main" org.apache.spark.sql.

Re: Adding support for truncate operator

My bad. I did not specify that I meant truncate operator on a column similar to how other maths operators work. On Sun, Oct 25, 2015 at 9:36 PM, Ted Yu wrote: > Have you seen the following ? > [SPARK-3907][SQL] Add truncate table support > > Cheers > > On Sun, Oct 25, 2015

Adding support for truncate operator

Hi! I noticed that SparkSQL does not support truncate operator as of now. Can we add it? I am willing to send over a PR for it

Re: Exception when using cosh

@Reynold submitted the PR: https://github.com/apache/spark/pull/9199 On Wed, Oct 21, 2015 at 11:01 PM, Shagun Sodhani wrote: > Sure! Would do that. > > Thanks a lot > > On Wed, Oct 21, 2015 at 10:59 PM, Reynold Xin wrote: > >> I think we made a mistake and forgot to

Re: Exception when using cosh

/analysis/FunctionRegistry.scala > > Do you mind submitting a pull request to fix this? Should be an one line > change. I filed a ticket to track this: > https://issues.apache.org/jira/browse/SPARK-11233 > > > > > On Wed, Oct 21, 2015 at 2:30 AM, Shagun Sodhani > wrote: > &g

Exception when using cosh

Hi! I was trying out different arithmetic functions in SparkSql. I noticed a weird thing. While *sinh* and *tanh* functions are working, using *cosh* results in an error saying: *Exception in thread "main" org.apache.spark.sql.AnalysisException: undefined function cosh;* The documentation says *c