On Thu, Nov 6, 2014 at 3:47 AM, Akila Ravihansa Perera <raviha...@wso2.com> wrote:
> Hi Nirmal, > > Great finding! > > But I don't follow your reasoning for this behavior. Could you explain how > this can be related to "less number of events" being published by agents? > > IMHO, events coming from agents should be sampled and averaged within a > fixed time intervals. This should be done in a separate CEP execution plan. > Those sampled events then should be sent to > SecondDerivativeFinderWindowProcessor. > In that way we won't run into issues where time intervals are less than > 1000 since it will take those values for pre-defined intervals. > > I might be missing a key piece here, but thought of sharing this :) > Thanks Akila! I think what you are missing here is, what we take into account is the real timestamp values of the events. Imagine scenario where we are calculating gradient for a cluster which spans few members. The agents of those members are sending events in a desynchronized manner. If you take a 60second interval and if you take the events by timestamp, you would see a graph like below; |*.. . . .... ... . * | In this case, for a more accurate value, currently we do not assume time gap is 60s, instead we take the real timestamp differences of the first and last event. This is a very simple calculation of gradient, and I think it's sufficient for our needs. Then in order to find second derivative; we would select another two events if possible. |*.. . . .... ... . * | this is a two step process, where we would calculate gradient from red dotted events and green dotted events separately and get the timestamp of the new gradient values as (t1+t2)/2. Then we again find the gradient from the two new gradient events and return it as the second derivative value. So, when there're less number of events and when it occurs in a sequence like below, we would need to normalize before calculating gradient.(t(e3) - t(e2) < 1s) |*. ..* | > Thanks. > -- Best Regards, Nirmal Nirmal Fernando. PPMC Member & Committer of Apache Stratos, Senior Software Engineer, WSO2 Inc. Blog: http://nirmalfdo.blogspot.com/
