Re: Calculating rolling average using aggregations

2014-09-19 Thread Michaƫl Gallego
+1 for this feature! What I need is pretty similar: calculate rolling sum, so for each day, I need to sum the previous 30 days (on each point). Oracle and Postgre make this very easy with aggregation function (and they can take advantage of very interesting optimization for sum, as each point

Re: Calculating rolling average using aggregations

2014-03-13 Thread Clinton Gormley
I rethought this problem last night. The solutions I've presented already are a lot less efficient than they could be, as they increase the work per doc by a factor of the number of buckets (ie 24h * 28d = 672). It'd be much more efficient to calculate this rolling average client side in a single

Re: Calculating rolling average using aggregations

2014-03-12 Thread Simon Cast
Thanks Clinton. That looks interesting but that creates an average of values per hour for the last 28 days. What I am looking for more precisely is to calculate a 28 rolling (or moving average) using the last 28 days of data and redoing that calculation every hour. I suppose what could be

Re: Calculating rolling average using aggregations

2014-03-12 Thread Clinton Gormley
Forgive my ignorance but could you provide a worked example of what you mean? I'm not getting exactly what you're after. On 12 March 2014 13:24, Simon Cast simon.c...@gmail.com wrote: Thanks Clinton. That looks interesting but that creates an average of values per hour for the last 28 days.

Re: Calculating rolling average using aggregations

2014-03-12 Thread Binh Ly
Unfortunately a MA aggregation is not available at the moment (unless you accumulate/include all the prior data that you need into every doc, or you run multiple queries and aggregate it yourself - neither of which is desirable or flexible) Clint, a MA is an average computed in a sliding

Re: Calculating rolling average using aggregations

2014-03-12 Thread Clinton Gormley
Heya Bihn The part I'm not getting is this: the rolling average for every hour in the last 28 days. ie what period should each bucket/rolling avg cover? an hour? 28 days? You can still do rolling averages with aggregations, but they require a bit more work. I wanted to get the exact specs before

Re: Calculating rolling average using aggregations

2014-03-12 Thread Clinton Gormley
OK, Binh took me aside and explained what you are after: a number per hour which reflects the average of the previous 28*24 hours. There are two ways you can do this with aggregations. The first way is to create overlapping buckets, so each date will fall into multiple windows, eg: Jan 1,

Calculating rolling average using aggregations

2014-03-11 Thread Simon Cast
Hi, I'm looking at using the new aggregations module of Elasticsearch to produce some metrics for my application. I want to calculate the rolling average for every hour in the last 28 days and then plot these on a chart. I can see how this could be done using a query per hour for the last 28