Custom UDAF Evaluated Over Window

2015-09-10 Thread xander92
While testing out the new UserDefinedAggregateFunction in Spark 1.5.0, I successfully implemented a simple function to compute an average. I then tried to test this function by applying it over a simple window and I got an error saying that my function is not supported over window operation. So,

How to evaluate custom UDF over window

2015-08-24 Thread xander92
The ultimate aim of my program is to be able to wrap an arbitrary Scala function (mostly will be statistics / customized rolling window metrics) in a UDF and evaluate them on DataFrames using the window functionality. So my main question is how do I express that a UDF takes a Frame of rows from a