Re: [Spark-SQL] Custom aggregate function for GrouppedData

Thanks Michael for replying, Aggregator/UDAF is exactly what I am looking for, but are still on 1.4 and it's gonna take time to get 1.6. On Wed, Jan 6, 2016 at 10:32 AM, Michael Armbrust wrote: > In Spark 1.6 GroupedDataset >

Re: [Spark-SQL] Custom aggregate function for GrouppedData

In Spark 1.6 GroupedDataset has mapGroups, which sounds like what you are looking for. You can also write a custom Aggregator

[Spark-SQL] Custom aggregate function for GrouppedData

Hello Hivemind, Referring to this thread - https://forums.databricks.com/questions/956/how-do-i-group-my-dataset-by-a-key-or-combination.html. I have learnt that we can not do much with groupped data apart from using existing aggregate functions. This blog post was written in may 2015, I don't kno