On Sat, Nov 8, 2014 at 12:49 AM, Nirmal Fernando <nirmal070...@gmail.com> wrote:
> Hi Lahiru, > > All these are improvements and should have discussed in a new thread IMO. > (these are buried now) > +1. I will send a complete proposal to a new thread. Thanks. > > What we currently have is linear gradient, which is the simplest form > (first step!) > > On Thu, Nov 6, 2014 at 2:45 PM, Lahiru Sandaruwan <lahi...@wso2.com> > wrote: > >> Hi all, >> >> Very interesting discussion and thanks Nirmal for initiating this! >> >> I also did some research on this. >> I think best approach to find gradient and second derivative is to "curve >> fitting". Refer "Locally weighted linear regression" section at [1]. We >> should be calculating gradient and second derivative by differentiating the >> equation of fitted line. >> >> We would need a second degree polynomial fitter for this, where we can >> use Apache commons math library for this. Refer the sample at [2], we need >> to run this with degree 2 instead of 3. >> >> E.g. >> So if get degree 2 polynomial fitter, we will have an equation like below >> where value(v) is our statistic value, time(t) is the time of event, >> gradient(g), and second derivative(sd). >> >> Equation we get from received events, >> v = a*t*t + b*t + c >> >> g = 2*a*t >> > > g= 2*a*t + b > > >> sd = 2*a >> >> We can get *g* by applying the median of the time stamp sample. >> > > If you use median of the timestamp sample, g would grow with time?! > > >> >> I have run a sample successfully. I can complete the work on CEP >> extension if you guys agree. >> >> In addition to this, we should also restrict the sample size as Akila >> suggested. Ideally we should receive 6 events per minute for memberwise >> stats and "6 * no. of instances" for clusterwise stats. >> > > From where did you get '6' ? > > >> We should put a constraint with this much of percentage(say 80%) should >> be there to calculate *g *and* sd.* >> > > This much of a percentage from what value? > >> >> Also we should not calculated gradient of stats sent from different >> members in clusterwise scenario as Imesh mentioned. That causes this type >> of small time gap issues. >> >> Thanks. >> >> [1] http://cs229.stanford.edu/notes/cs229-notes1.pdf >> [2] http://commons.apache.org/proper/commons-math/userguide/fitting.html >> >> On Thu, Nov 6, 2014 at 6:41 PM, Imesh Gunaratne <im...@apache.org> wrote: >> >>> Raj: Thanks for the clarification, it was my misunderstanding. >>> >>> I did further analysis on this issue, seems like the root cause of this >>> problem resides somewhere else. >>> >>> Currently we have set the cartridge agent's statistics publishing >>> interval to 15 seconds. If so how do we receive series of events within a >>> second? Seems like we are considering statistics events coming from >>> different members for calculating the gradient. This might not be correct, >>> if we were to draw a curve of a data set and consider the gradient of those >>> values, those values may have come from the same source. Otherwise the >>> gradient value we are calculating might not be correct. >>> >>> Thanks >>> >>> On Thu, Nov 6, 2014 at 2:05 PM, Udara Liyanage <ud...@wso2.com> wrote: >>> >>>> Hi, >>>> >>>> Imagine we put the values in a graph (value against time). The gradient >>>> is the slope of the line. When we draw the line via two very near dots, we >>>> get a high slope/gradient even for a low increase of the values. This >>>> happens when we have only events which are occurred in very near time >>>> intervals. >>>> >>>> On the other hands, when events are distributed among long time period >>>> (are more counts in sample) we don't get this high gradient since the line >>>> goes via higher dots count where we get a reasonable value for gradient. >>>> >>>> As I understood the scenario, it is reasonable to make time gap equals >>>> to millisInSeconds in case time gap is very small. >>>> >>>> Touched, not typed. Erroneous words are a feature, not a typo. >>>> I'd rather keep it simple Akila (this would complex the scenario of >>>> second derivative too) :-) Isn't it? >>>> >>>> On Thu, Nov 6, 2014 at 9:03 AM, Akila Ravihansa Perera < >>>> raviha...@wso2.com> wrote: >>>> >>>>> Until t2 - t1 > 1 >>>>> >>>>> On Thu, Nov 6, 2014 at 12:50 AM, Nirmal Fernando < >>>>> nirmal070...@gmail.com> wrote: >>>>> >>>>>> >>>>>> >>>>>> On Thu, Nov 6, 2014 at 8:33 AM, Akila Ravihansa Perera < >>>>>> raviha...@wso2.com> wrote: >>>>>> >>>>>>> Thanks for the explanation, Nirmal. It makes sense now. >>>>>>> >>>>>>> Perhaps as an alternative solution, window processor can be set to >>>>>>> wait until sufficient events have arrived to calculate the gradient? >>>>>>> >>>>>> >>>>>> What would be the sufficient value? Theoretically in gradient case >>>>>> it's 2, in second derivative case it's 3. And we're doing that already. >>>>>> >>>>>>> >>>>>>> On Wed, Nov 5, 2014 at 11:48 PM, Rajkumar Rajaratnam < >>>>>>> rajkum...@wso2.com> wrote: >>>>>>> >>>>>>>> In other words, fix reduces the risks which are coming from bad >>>>>>>> samples. >>>>>>>> >>>>>>>> On Thu, Nov 6, 2014 at 12:08 PM, Rajkumar Rajaratnam < >>>>>>>> rajkum...@wso2.com> wrote: >>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Thu, Nov 6, 2014 at 11:38 AM, Imesh Gunaratne <im...@apache.org >>>>>>>>> > wrote: >>>>>>>>> >>>>>>>>>> On Thu, Nov 6, 2014 at 9:22 AM, Rajkumar Rajaratnam < >>>>>>>>>> rajkum...@wso2.com> wrote: >>>>>>>>>> >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> No Imesh. There are no difference between *taking the the >>>>>>>>>>> difference and dividing it by 1000* and *dividing it by 1000 >>>>>>>>>>> and taking the difference*. >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Raj: I agree, in this scenario it makes no difference, however >>>>>>>>>> when we are applying a fomular the unit conversion should happen >>>>>>>>>> prior to >>>>>>>>>> applying values. >>>>>>>>>> >>>>>>>>> >>>>>>>>> Not exactly Imesh. We can do operations if operands are in the >>>>>>>>> same units. So we can do subtract operation in milliseconds. And then >>>>>>>>> we >>>>>>>>> can do division operation to make it in seconds. On the other hand, >>>>>>>>> we can >>>>>>>>> do a division operation to make each operands to seconds, then since >>>>>>>>> both >>>>>>>>> are seconds, we can do a subtract operation. So we can apply the >>>>>>>>> formula if >>>>>>>>> operands are in same unit. I guess this is same for all scenarios. >>>>>>>>> >>>>>>>>> >>>>>>>>>> Still I see a problem with this correction, take the same example: >>>>>>>>>> Gradient: -999.9999999999998 Last val: 7.000000000000001 First >>>>>>>>>> val: 12.0 Time Gap: 5 t1: 1415213232152 t2: 1415213232157 >>>>>>>>>> >>>>>>>>>> *Manual calulation:* >>>>>>>>>> The value difference: 7 - 12 = -5 >>>>>>>>>> Time difference: 1415213232157 - 1415213232152 = 5 ms => 5/1000 = >>>>>>>>>> 0.005 sec >>>>>>>>>> *Gradient = - 5/0.005 = -1000* >>>>>>>>>> >>>>>>>>>> *According to the current fix:* >>>>>>>>>> long millisecondsForASecond = 1000; >>>>>>>>>> long tGap = t2 - t1 > millisecondsForASecond ? t2 - t1 : >>>>>>>>>> millisecondsForASecond; >>>>>>>>>> double gradient = 0.0; >>>>>>>>>> if (tGap > 0) { >>>>>>>>>> gradient = ((lastVal - firstVal) * >>>>>>>>>> millisecondsForASecond) / tGap; >>>>>>>>>> } >>>>>>>>>> >>>>>>>>>> t1: 1415213232152 t2: 1415213232157 firstVal: 12 lastVall: 7 >>>>>>>>>> Gradient = -5 * 1000 / 1000 = - 5 >>>>>>>>>> >>>>>>>>> >>>>>>>>> IMO, this is a sufficient solution. >>>>>>>>> >>>>>>>>> Let me explain how I see Nirmal's solution. >>>>>>>>> >>>>>>>>> We are taking only first and last values in a window for gradient >>>>>>>>> calculation. If there are more events in that window, and if that is >>>>>>>>> a good >>>>>>>>> sample, tGap will be taking some higher values than 1s. This is usual >>>>>>>>> situation. Nirmal's fix is nothing to do with this scenario. >>>>>>>>> >>>>>>>>> If there are few events in that window(say only 2), they can be >>>>>>>>> really nearer. So there is a possibility that tGap can take a value >>>>>>>>> near to >>>>>>>>> 0. In that case gradient will be a very big value. Hence, if the tGap >>>>>>>>> is >>>>>>>>> very low (less than 1 second), then we can assume that the first and >>>>>>>>> last >>>>>>>>> events actually occurred in 1 second gap. The reason why tGap is less >>>>>>>>> than >>>>>>>>> 0 is, there are only few events in that window, and that sample is >>>>>>>>> not good >>>>>>>>> enough. But this a rare situation, it can only happen once in a while. >>>>>>>>> Nirmal's fix is only dealing with this scenario. Fix imposes a lower >>>>>>>>> limit >>>>>>>>> in the time interval between two events. This fix is really important >>>>>>>>> to >>>>>>>>> deal with samples which are not good enough. >>>>>>>>> >>>>>>>>> By previous formula, we get -1000. This happened because the >>>>>>>>> sample is not good enough (may be there are few events). Fix makes it >>>>>>>>> to >>>>>>>>> -5. >>>>>>>>> >>>>>>>>> So my understanding is that the solution is good enough for now. >>>>>>>>> >>>>>>>>> Thanks. >>>>>>>>> >>>>>>>>>> >>>>>>>>>> Nirmal: Can you please justify this? >>>>>>>>>> >>>>>>>>>> Thanks >>>>>>>>>> >>>>>>>>>> On Thu, Nov 6, 2014 at 11:31 AM, Nirmal Fernando < >>>>>>>>>> nirmal070...@gmail.com> wrote: >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Thu, Nov 6, 2014 at 6:05 AM, Gayan Gunarathne < >>>>>>>>>>> gay...@wso2.com> wrote: >>>>>>>>>>> >>>>>>>>>>>> AFAIU this is not related with the gradient calculation formula >>>>>>>>>>>> that we used. But It is related with the how we calculate gradient >>>>>>>>>>>> when the >>>>>>>>>>>> denominator become a small number.IMO when the divisor >>>>>>>>>>>> (timeInSeconds) >>>>>>>>>>>> approaches to the small number we do need to normalize the result. >>>>>>>>>>>> >>>>>>>>>>>> IMO we do have max and min scale up and down values. So based >>>>>>>>>>>> on that we can calculate the gradient on that value range.Simply >>>>>>>>>>>> if the >>>>>>>>>>>> denominator approaches zero result approaches to infinity. >>>>>>>>>>>> >>>>>>>>>>>> I think that's what Nirmal doing here by making the lower >>>>>>>>>>>> threshold as "millisecondsForASecond" >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> You got it! thanks. >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> long tGap = t2 - t1 > millisecondsForASecond ? t2 - t1 >>>>>>>>>>>> : millisecondsForASecond; >>>>>>>>>>>> double gradient = 0.0; >>>>>>>>>>>> if (tGap > 0) { >>>>>>>>>>>> gradient = ((lastVal - firstVal) * >>>>>>>>>>>> millisecondsForASecond) / tGap; >>>>>>>>>>>> } >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Gayan >>>>>>>>>>>> >>>>>>>>>>>> On Thu, Nov 6, 2014 at 9:22 AM, Rajkumar Rajaratnam < >>>>>>>>>>>> rajkum...@wso2.com> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi, >>>>>>>>>>>>> >>>>>>>>>>>>> On Thu, Nov 6, 2014 at 9:03 AM, Imesh Gunaratne < >>>>>>>>>>>>> im...@apache.org> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hi Nirmal, >>>>>>>>>>>>>> >>>>>>>>>>>>>> I do not think the milliseconds to seconds convertion is >>>>>>>>>>>>>> correct here. >>>>>>>>>>>>>> >>>>>>>>>>>>>> As I see we are taking the difference of two timestamp values >>>>>>>>>>>>>> and then dividing it by 1000. The corect way might be to first >>>>>>>>>>>>>> divide each >>>>>>>>>>>>>> value by 1000 and then take the difference. >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> No Imesh. There are no difference between *taking the the >>>>>>>>>>>>> difference and dividing it by 1000* and *dividing it by 1000 >>>>>>>>>>>>> and taking the difference*. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> On the other hand we might not need to convert these values >>>>>>>>>>>>>> to seconds since we are taking a time difference and calculating >>>>>>>>>>>>>> a gradient. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I did a quick test with the following sample: >>>>>>>>>>>>>> Gradient: -999.9999999999998 Last val: 7.000000000000001 >>>>>>>>>>>>>> First val: 12.0 Time Gap: 5 t1: 1415213232152 t2: 1415213232157 >>>>>>>>>>>>>> >>>>>>>>>>>>>> *According previous code:* >>>>>>>>>>>>>> long tGap = t2 - t1; >>>>>>>>>>>>>> double gradient = 0.0; >>>>>>>>>>>>>> if (tGap > 0) { >>>>>>>>>>>>>> gradient = ((lastVal - firstVal) * 1000) / tGap; >>>>>>>>>>>>>> } >>>>>>>>>>>>>> >>>>>>>>>>>>>> t1: 1415213232152 t2: 1415213232157 firstVal: 12 lastVall: 7 >>>>>>>>>>>>>> gradient: -1000.0 >>>>>>>>>>>>>> >>>>>>>>>>>>>> *According to your fix:* >>>>>>>>>>>>>> long millisecondsForASecond = 1000; >>>>>>>>>>>>>> long tGap = t2 - t1 > millisecondsForASecond ? t2 - >>>>>>>>>>>>>> t1 : millisecondsForASecond; >>>>>>>>>>>>>> double gradient = 0.0; >>>>>>>>>>>>>> if (tGap > 0) { >>>>>>>>>>>>>> gradient = ((lastVal - firstVal) * >>>>>>>>>>>>>> millisecondsForASecond) / tGap; >>>>>>>>>>>>>> } >>>>>>>>>>>>>> >>>>>>>>>>>>>> t1: 1415213232152 t2: 1415213232157 firstVal: 12 lastVall: 7 >>>>>>>>>>>>>> gradient: -5.0 >>>>>>>>>>>>>> >>>>>>>>>>>>>> *According to an online gradient calculator:* >>>>>>>>>>>>>> gradient: -1 >>>>>>>>>>>>>> >>>>>>>>>>>>>> http://www.calculator.net/slope-calculator.html?type=1&x11=1415213232152&y11=12&x12=1415213232157&y12=7&x=27&y=19 >>>>>>>>>>>>>> >>>>>>>>>>>>>> According to the online gradient calculator (assuming their >>>>>>>>>>>>>> calculation is correct), the calculation in your fix is not >>>>>>>>>>>>>> correct. I >>>>>>>>>>>>>> believe the logic should be simple as follows: >>>>>>>>>>>>>> >>>>>>>>>>>>>> long tGap = t2 - t1; >>>>>>>>>>>>>> double gradient = 0.0; >>>>>>>>>>>>>> if (tGap > 0) { >>>>>>>>>>>>>> gradient = ((lastVal - firstVal)) / tGap; >>>>>>>>>>>>>> } >>>>>>>>>>>>>> >>>>>>>>>>>>>> t1: 1415213232152 t2: 1415213232157 firstVal: 12 lastVall: 7 >>>>>>>>>>>>>> gradient: -1.0 >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> And AFAIK, standard way to calculate gradient = >>>>>>>>>>>>> (v1-v2)/timeInSeconds. >>>>>>>>>>>>> >>>>>>>>>>>>> Well, we can divide it by what ever time units we wants, but >>>>>>>>>>>>> the result's unit will be different. >>>>>>>>>>>>> >>>>>>>>>>>>> In Nirmal's case, he get will get *stats/seconds*. In your >>>>>>>>>>>>> case, you will get *stats/milliseconds* >>>>>>>>>>>>> >>>>>>>>>>>>> And AS prediction is based on *seconds*, CEP should use >>>>>>>>>>>>> *seconds* for the calculation. >>>>>>>>>>>>> >>>>>>>>>>>>> wdyt? >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Thu, Nov 6, 2014 at 1:32 AM, Nirmal Fernando < >>>>>>>>>>>>>> nirmal070...@gmail.com> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi Guys, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> So, I got a chance to reproduce and analyze this issue. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> How we calculate gradient of two events? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Say the events are; e1(t1,v1) and e2(t2,v2) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> tx - time in *milliseconds* when xth event occurred >>>>>>>>>>>>>>> vx - value (memory, cpu etc.) that xth event carries >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> *time gap = t(2-1) = t2 -t1 milliseconds* >>>>>>>>>>>>>>> time gap in seconds = t(2-1) = (t2 - t1)/1000 seconds >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hence, >>>>>>>>>>>>>>> *Gradient = (v2 - v1) / t(2-1) = ( (v2 - v1) * 1000 ) / >>>>>>>>>>>>>>> (t2 - t1)* >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I've enabled debug logs for CEP extension; >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> log4j.logger.org.apache.stratos.cep.extension=DEBUG >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Please find the following 3 logs extracted from the debug >>>>>>>>>>>>>>> logs; >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> =================================================================== >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-11-05 19:47:27,073] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.cep.extension.SecondDerivativeFinderWindowProcessor} >>>>>>>>>>>>>>> - >>>>>>>>>>>>>>> Gradient: -0.1996007984031936 Last val: 9.0 First val: 12.0 >>>>>>>>>>>>>>> *Time >>>>>>>>>>>>>>> Gap: 15030* t1: 1415213202095 t2: 1415213217125 hash: >>>>>>>>>>>>>>> 155426542 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-11-05 19:47:27,073] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.cep.extension.SecondDerivativeFinderWindowProcessor} >>>>>>>>>>>>>>> - Gradient: >>>>>>>>>>>>>>> -999.9999999999998 Last val: 7.000000000000001 First val: >>>>>>>>>>>>>>> 12.0 *Time Gap: 5* t1: 1415213232152 t2: 1415213232157 >>>>>>>>>>>>>>> hash: 155426542 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-11-05 19:47:27,074] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.cep.extension.SecondDerivativeFinderWindowProcessor} >>>>>>>>>>>>>>> - >>>>>>>>>>>>>>> Gradient: -44.34884666437174 Last val: -999.9999999999998 >>>>>>>>>>>>>>> First val: >>>>>>>>>>>>>>> -0.1996007984031936 *Time Gap: 22544* t1: 1415213209610 t2: >>>>>>>>>>>>>>> 1415213232154 hash: 155426542 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> =================================================================== >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> So, as you can see the reason behind a large value is when >>>>>>>>>>>>>>> the time gap between two subjected events is less than 1 >>>>>>>>>>>>>>> second. This could >>>>>>>>>>>>>>> happen since events are coming from different asynchronous >>>>>>>>>>>>>>> agents and also >>>>>>>>>>>>>>> when there are less number of events. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> So, the fix I propose is a very simple one and it will not >>>>>>>>>>>>>>> compromise anything AFAIS. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Fix is to calculate time gap as follows; >>>>>>>>>>>>>>> *____ >>>>>>>>>>>>>>> t2 - t1* >>>>>>>>>>>>>>> *| >>>>>>>>>>>>>>> yes?* >>>>>>>>>>>>>>> *time gap = t(2-1) = t2 -t1 > 1000 ---* >>>>>>>>>>>>>>> *|____ >>>>>>>>>>>>>>> 1000* >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I have tested this and works fine. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ---------- Forwarded message ---------- >>>>>>>>>>>>>>> From: Manula Chathurika Thantriwatte <manu...@wso2.com> >>>>>>>>>>>>>>> Date: Tue, Oct 28, 2014 at 5:16 AM >>>>>>>>>>>>>>> Subject: Re: CEP sends very large values for gradient and >>>>>>>>>>>>>>> second derivative of load average >>>>>>>>>>>>>>> To: "dev@stratos.apache.org" <dev@stratos.apache.org> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi Raj, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Yes it has. But it's very easy to find the values from the >>>>>>>>>>>>>>> agent with the time stamp. Then we can search that time stamp >>>>>>>>>>>>>>> in the CEP >>>>>>>>>>>>>>> trace logs and compare the summarized values. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks ! >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Mon, Oct 27, 2014 at 9:23 PM, Rajkumar Rajaratnam < >>>>>>>>>>>>>>> rajkum...@wso2.com> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Mon, Oct 27, 2014 at 7:56 PM, Manula Chathurika >>>>>>>>>>>>>>>> Thantriwatte <manu...@wso2.com> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Hi Raj, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Shall we enable the cartridge agent debug logs. Then we >>>>>>>>>>>>>>>>> can compare both CEP and agent logs with the time stamp. Then >>>>>>>>>>>>>>>>> we can narrow >>>>>>>>>>>>>>>>> down where the actual problem is. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Manula, cep-trace log have the values sent by cartridge >>>>>>>>>>>>>>>> agent also right? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks ! >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On Mon, Oct 27, 2014 at 5:06 PM, Rajkumar Rajaratnam < >>>>>>>>>>>>>>>>> rajkum...@wso2.com> wrote: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On Mon, Oct 27, 2014 at 12:53 PM, Manula Chathurika >>>>>>>>>>>>>>>>>> Thantriwatte <manu...@wso2.com> wrote: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Hi Raj, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Is this happen frequently or intermittently ? Once I go >>>>>>>>>>>>>>>>>>> though the logs I'm unable to find the high input values >>>>>>>>>>>>>>>>>>> for load average. >>>>>>>>>>>>>>>>>>> But output have high values. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> 22:10:38,698 [-] [Siddhi-Scheduler-pool-10-thread-1] >>>>>>>>>>>>>>>>>>> INFO EVENT_TRACE_LOGGER TenantId=-1234 : Output Event >>>>>>>>>>>>>>>>>>> Adaptor : >>>>>>>>>>>>>>>>>>> JMSOutputAdaptor, sent >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> {"org.apache.stratos.messaging.event.health.stat.SecondDerivativeOfLoadAverageEvent":{"message":{"clusterId":"php.php.domain","networkPartitionId":"","value":"2000000.0"}}} >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> This is happening intermittently. Experienced now too. >>>>>>>>>>>>>>>>>> This is a serious issue. We need to somehow sort this out. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Also networkPartitionId is empty. In the current >>>>>>>>>>>>>>>>>>> execution plans it took networkPartitionId for the >>>>>>>>>>>>>>>>>>> calculation. IMO we need >>>>>>>>>>>>>>>>>>> to update the CEP execution plans for 4.1.0. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Thanks ! >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On Sun, Oct 26, 2014 at 8:49 AM, Manula Chathurika >>>>>>>>>>>>>>>>>>> Thantriwatte <manu...@wso2.com> wrote: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Thanks Raj. I'll go through them. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Thanks ! >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On Sun, Oct 26, 2014 at 8:33 AM, Rajkumar Rajaratnam < >>>>>>>>>>>>>>>>>>>> rajkum...@wso2.com> wrote: >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Hi Manula, >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> You can see those values in the same log I attached in >>>>>>>>>>>>>>>>>>>>> the previous mail. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Thanks. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> On Sun, Oct 26, 2014 at 8:29 AM, Rajkumar Rajaratnam < >>>>>>>>>>>>>>>>>>>>> rajkum...@wso2.com> wrote: >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Hi Chamila, >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> I couldn't find any large values sent by agent. There >>>>>>>>>>>>>>>>>>>>>> are traces for CEP sending large values. Find the cep >>>>>>>>>>>>>>>>>>>>>> trace log here [1] >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> 1. >>>>>>>>>>>>>>>>>>>>>> https://drive.google.com/file/d/0B1haIleqJMHIS3FFVFpfa0JlbVE/view?usp=sharing >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Thanks. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> On Sun, Oct 26, 2014 at 8:23 AM, Manula Chathurika >>>>>>>>>>>>>>>>>>>>>> Thantriwatte <manu...@wso2.com> wrote: >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Shall we enable the cartridge agent debug logs and >>>>>>>>>>>>>>>>>>>>>>> see what are the load average values sent from the >>>>>>>>>>>>>>>>>>>>>>> agent. If those values >>>>>>>>>>>>>>>>>>>>>>> are normal then we can narrow down to the CEP. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Thanks ! >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> On Sat, Oct 25, 2014 at 10:43 PM, Chamila De Alwis < >>>>>>>>>>>>>>>>>>>>>>> chami...@wso2.com> wrote: >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Hi Raj, >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Can you inspect the CEP trace logs for the same >>>>>>>>>>>>>>>>>>>>>>>> time period? It will have the events published from >>>>>>>>>>>>>>>>>>>>>>>> the agent. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>>>>>>>>>>>> Chamila de Alwis >>>>>>>>>>>>>>>>>>>>>>>> Software Engineer | WSO2 | +94772207163 >>>>>>>>>>>>>>>>>>>>>>>> Blog: code.chamiladealwis.com >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> On Sat, Oct 25, 2014 at 10:39 PM, Rajkumar >>>>>>>>>>>>>>>>>>>>>>>> Rajaratnam <rajkum...@wso2.com> wrote: >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> I am doing a test round with M3 with python agent. >>>>>>>>>>>>>>>>>>>>>>>>> And faced this very strange scenario, $Subject. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,693] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} >>>>>>>>>>>>>>>>>>>>>>>>> - Load average >>>>>>>>>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub >>>>>>>>>>>>>>>>>>>>>>>>> cluster] KubGrp1 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,693] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>>>>>>>>>>>> - Grad of >>>>>>>>>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] >>>>>>>>>>>>>>>>>>>>>>>>> -2000.0 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,694] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} >>>>>>>>>>>>>>>>>>>>>>>>> - Load average >>>>>>>>>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub >>>>>>>>>>>>>>>>>>>>>>>>> cluster] KubGrp1 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,694] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>>>>>>>>>>>> - Grad of >>>>>>>>>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 0.0 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,694] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} >>>>>>>>>>>>>>>>>>>>>>>>> - Load average >>>>>>>>>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub >>>>>>>>>>>>>>>>>>>>>>>>> cluster] KubGrp1 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,694] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>>>>>>>>>>>> - Avg >>>>>>>>>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] >>>>>>>>>>>>>>>>>>>>>>>>> 1.9090909 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,694] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} >>>>>>>>>>>>>>>>>>>>>>>>> - Load average >>>>>>>>>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub >>>>>>>>>>>>>>>>>>>>>>>>> cluster] KubGrp1 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,694] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>>>>>>>>>>>> - Avg >>>>>>>>>>>>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain >>>>>>>>>>>>>>>>>>>>>>>>> [value] 28.718182 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,695] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>>>>>>>>>>>> - Second >>>>>>>>>>>>>>>>>>>>>>>>> Derivation of Memory Consumption event: [cluster] >>>>>>>>>>>>>>>>>>>>>>>>> php.php.domain [value] >>>>>>>>>>>>>>>>>>>>>>>>> 200000.0 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,695] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>>>>>>>>>>>> - Grad of >>>>>>>>>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 0.0 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,695] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} >>>>>>>>>>>>>>>>>>>>>>>>> - Load average >>>>>>>>>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub >>>>>>>>>>>>>>>>>>>>>>>>> cluster] KubGrp1 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,695] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>>>>>>>>>>>> - Second >>>>>>>>>>>>>>>>>>>>>>>>> Derivation of load avg event: [cluster] >>>>>>>>>>>>>>>>>>>>>>>>> php.php.domain [value] 0.0 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,695] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} >>>>>>>>>>>>>>>>>>>>>>>>> - Load average >>>>>>>>>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub >>>>>>>>>>>>>>>>>>>>>>>>> cluster] KubGrp1 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,696] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>>>>>>>>>>>> - Second >>>>>>>>>>>>>>>>>>>>>>>>> Derivation of load avg event: [cluster] >>>>>>>>>>>>>>>>>>>>>>>>> php.php.domain [value] 0.0 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,696] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} >>>>>>>>>>>>>>>>>>>>>>>>> - Load average >>>>>>>>>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub >>>>>>>>>>>>>>>>>>>>>>>>> cluster] KubGrp1 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,696] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>>>>>>>>>>>> - Avg >>>>>>>>>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] >>>>>>>>>>>>>>>>>>>>>>>>> 3.3636363 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,696] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} >>>>>>>>>>>>>>>>>>>>>>>>> - Load average >>>>>>>>>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub >>>>>>>>>>>>>>>>>>>>>>>>> cluster] KubGrp1 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,696] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>>>>>>>>>>>> - Grad of >>>>>>>>>>>>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain >>>>>>>>>>>>>>>>>>>>>>>>> [value] 0.11670045 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,696] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} >>>>>>>>>>>>>>>>>>>>>>>>> - Memory >>>>>>>>>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check >>>>>>>>>>>>>>>>>>>>>>>>> [kub cluster] KubGrp1 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>>>>>>>>>>>> - Avg >>>>>>>>>>>>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain >>>>>>>>>>>>>>>>>>>>>>>>> [value] 28.7 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} >>>>>>>>>>>>>>>>>>>>>>>>> - Memory >>>>>>>>>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check >>>>>>>>>>>>>>>>>>>>>>>>> [kub cluster] KubGrp1 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>>>>>>>>>>>> - Second >>>>>>>>>>>>>>>>>>>>>>>>> Derivation of Memory Consumption event: [cluster] >>>>>>>>>>>>>>>>>>>>>>>>> php.php.domain [value] 0.0 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} >>>>>>>>>>>>>>>>>>>>>>>>> - Memory >>>>>>>>>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check >>>>>>>>>>>>>>>>>>>>>>>>> [kub cluster] KubGrp1 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>>>>>>>>>>>> - Avg >>>>>>>>>>>>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain >>>>>>>>>>>>>>>>>>>>>>>>> [value] 28.7 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} >>>>>>>>>>>>>>>>>>>>>>>>> - Memory >>>>>>>>>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check >>>>>>>>>>>>>>>>>>>>>>>>> [kub cluster] KubGrp1 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>>>>>>>>>>>> - Avg >>>>>>>>>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 5.0 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} >>>>>>>>>>>>>>>>>>>>>>>>> - Load average >>>>>>>>>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub >>>>>>>>>>>>>>>>>>>>>>>>> cluster] KubGrp1 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>>>>>>>>>>>> - Grad of >>>>>>>>>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] >>>>>>>>>>>>>>>>>>>>>>>>> 2000.0 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,698] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} >>>>>>>>>>>>>>>>>>>>>>>>> - Load average >>>>>>>>>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub >>>>>>>>>>>>>>>>>>>>>>>>> cluster] KubGrp1 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,698] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>>>>>>>>>>>> - Avg >>>>>>>>>>>>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain >>>>>>>>>>>>>>>>>>>>>>>>> [value] 28.7 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,698] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} >>>>>>>>>>>>>>>>>>>>>>>>> - Memory >>>>>>>>>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check >>>>>>>>>>>>>>>>>>>>>>>>> [kub cluster] KubGrp1 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,698] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>>>>>>>>>>>> - Grad of >>>>>>>>>>>>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain >>>>>>>>>>>>>>>>>>>>>>>>> [value] 0.0 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,698] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} >>>>>>>>>>>>>>>>>>>>>>>>> - Memory >>>>>>>>>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check >>>>>>>>>>>>>>>>>>>>>>>>> [kub cluster] KubGrp1 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,699] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>>>>>>>>>>>> - Second >>>>>>>>>>>>>>>>>>>>>>>>> Derivation of Memory Consumption event: [cluster] >>>>>>>>>>>>>>>>>>>>>>>>> php.php.domain [value] 0.0 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,699] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} >>>>>>>>>>>>>>>>>>>>>>>>> - Memory >>>>>>>>>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check >>>>>>>>>>>>>>>>>>>>>>>>> [kub cluster] KubGrp1 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,700] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>>>>>>>>>>>> - Second >>>>>>>>>>>>>>>>>>>>>>>>> Derivation of load avg event: [cluster] >>>>>>>>>>>>>>>>>>>>>>>>> php.php.domain [value] 2000000.0 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,700] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} >>>>>>>>>>>>>>>>>>>>>>>>> - Load average >>>>>>>>>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub >>>>>>>>>>>>>>>>>>>>>>>>> cluster] KubGrp1 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,700] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>>>>>>>>>>>> - Grad of >>>>>>>>>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] >>>>>>>>>>>>>>>>>>>>>>>>> 13500.0 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,700] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} >>>>>>>>>>>>>>>>>>>>>>>>> - Load average >>>>>>>>>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub >>>>>>>>>>>>>>>>>>>>>>>>> cluster] KubGrp1 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,701] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>>>>>>>>>>>> - Grad of >>>>>>>>>>>>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain >>>>>>>>>>>>>>>>>>>>>>>>> [value] 50.0 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,701] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} >>>>>>>>>>>>>>>>>>>>>>>>> - Memory >>>>>>>>>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check >>>>>>>>>>>>>>>>>>>>>>>>> [kub cluster] KubGrp1 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,701] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>>>>>>>>>>>> - Avg >>>>>>>>>>>>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain >>>>>>>>>>>>>>>>>>>>>>>>> [value] 28.738462 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,701] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} >>>>>>>>>>>>>>>>>>>>>>>>> - Memory >>>>>>>>>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check >>>>>>>>>>>>>>>>>>>>>>>>> [kub cluster] KubGrp1 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,702] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>>>>>>>>>>>> - Avg >>>>>>>>>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] >>>>>>>>>>>>>>>>>>>>>>>>> 2.3333333 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,702] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} >>>>>>>>>>>>>>>>>>>>>>>>> - Load average >>>>>>>>>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub >>>>>>>>>>>>>>>>>>>>>>>>> cluster] KubGrp1 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,702] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>>>>>>>>>>>> - Grad of >>>>>>>>>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 0.0 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,702] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} >>>>>>>>>>>>>>>>>>>>>>>>> - Load average >>>>>>>>>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub >>>>>>>>>>>>>>>>>>>>>>>>> cluster] KubGrp1 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,703] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>>>>>>>>>>>> - Grad of >>>>>>>>>>>>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain >>>>>>>>>>>>>>>>>>>>>>>>> [value] 0.0 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,703] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} >>>>>>>>>>>>>>>>>>>>>>>>> - Memory >>>>>>>>>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check >>>>>>>>>>>>>>>>>>>>>>>>> [kub cluster] KubGrp1 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,703] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>>>>>>>>>>>> - Avg >>>>>>>>>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 31.0 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,703] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} >>>>>>>>>>>>>>>>>>>>>>>>> - Load average >>>>>>>>>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub >>>>>>>>>>>>>>>>>>>>>>>>> cluster] KubGrp1 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,704] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>>>>>>>>>>>> - Grad of >>>>>>>>>>>>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain >>>>>>>>>>>>>>>>>>>>>>>>> [value] 0.0 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,704] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} >>>>>>>>>>>>>>>>>>>>>>>>> - Memory >>>>>>>>>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check >>>>>>>>>>>>>>>>>>>>>>>>> [kub cluster] KubGrp1 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,704] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>>>>>>>>>>>> - Avg >>>>>>>>>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 39.0 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,704] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} >>>>>>>>>>>>>>>>>>>>>>>>> - Load average >>>>>>>>>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub >>>>>>>>>>>>>>>>>>>>>>>>> cluster] KubGrp1 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,704] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>>>>>>>>>>>> - Avg >>>>>>>>>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 39.0 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,705] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} >>>>>>>>>>>>>>>>>>>>>>>>> - Load average >>>>>>>>>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub >>>>>>>>>>>>>>>>>>>>>>>>> cluster] KubGrp1 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,706] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>>>>>>>>>>>> - Grad of >>>>>>>>>>>>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] >>>>>>>>>>>>>>>>>>>>>>>>> -11000.0 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,706] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} >>>>>>>>>>>>>>>>>>>>>>>>> - Load average >>>>>>>>>>>>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub >>>>>>>>>>>>>>>>>>>>>>>>> cluster] KubGrp1 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,708] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>>>>>>>>>>>> - Grad of >>>>>>>>>>>>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain >>>>>>>>>>>>>>>>>>>>>>>>> [value] 0.0 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,708] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} >>>>>>>>>>>>>>>>>>>>>>>>> - Memory >>>>>>>>>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check >>>>>>>>>>>>>>>>>>>>>>>>> [kub cluster] KubGrp1 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,709] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>>>>>>>>>>>> - Avg >>>>>>>>>>>>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain >>>>>>>>>>>>>>>>>>>>>>>>> [value] 28.8 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,709] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} >>>>>>>>>>>>>>>>>>>>>>>>> - Memory >>>>>>>>>>>>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check >>>>>>>>>>>>>>>>>>>>>>>>> [kub cluster] KubGrp1 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,811] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.cloud.controller.util.PodActivationWatcher} >>>>>>>>>>>>>>>>>>>>>>>>> - >>>>>>>>>>>>>>>>>>>>>>>>> PodActivationWatcher running : Running >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,942] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.cloud.controller.util.PodActivationWatcher} >>>>>>>>>>>>>>>>>>>>>>>>> - >>>>>>>>>>>>>>>>>>>>>>>>> PodActivationWatcher running : Running >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,945] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.cloud.controller.util.PodActivationWatcher} >>>>>>>>>>>>>>>>>>>>>>>>> - >>>>>>>>>>>>>>>>>>>>>>>>> PodActivationWatcher running : Running >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,712] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesServiceClusterMonitor} >>>>>>>>>>>>>>>>>>>>>>>>> - >>>>>>>>>>>>>>>>>>>>>>>>> KubernetesServiceClusterMonitor is running.. >>>>>>>>>>>>>>>>>>>>>>>>> KubernetesServiceClusterMonitor [ >>>>>>>>>>>>>>>>>>>>>>>>> kubernetesHostClusterId=KubGrp1, >>>>>>>>>>>>>>>>>>>>>>>>> clusterId=php.php.domain, serviceId=php] >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,713] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesServiceClusterMonitor} >>>>>>>>>>>>>>>>>>>>>>>>> - >>>>>>>>>>>>>>>>>>>>>>>>> Running min check for [kub-cluster] : KubGrp1 >>>>>>>>>>>>>>>>>>>>>>>>> [cluster] : php.php.domain >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,713] INFO >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - >>>>>>>>>>>>>>>>>>>>>>>>> Running minimum rule: >>>>>>>>>>>>>>>>>>>>>>>>> [kub-cluster] KubGrp1 [cluster] php.php.domain >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,714] INFO >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - >>>>>>>>>>>>>>>>>>>>>>>>> [min-check] [cluster] : >>>>>>>>>>>>>>>>>>>>>>>>> php.php.domain [Replicas] nonTerminated : 3 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,714] INFO >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - >>>>>>>>>>>>>>>>>>>>>>>>> [min-check] [cluster] : >>>>>>>>>>>>>>>>>>>>>>>>> php.php.domain [Replicas] minReplicas : 3 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,715] INFO >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - >>>>>>>>>>>>>>>>>>>>>>>>> Running obsolete containers >>>>>>>>>>>>>>>>>>>>>>>>> rule [kub-cluster] : KubGrp1 [cluster] : >>>>>>>>>>>>>>>>>>>>>>>>> php.php.domain >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,715] INFO >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - >>>>>>>>>>>>>>>>>>>>>>>>> [obsolete-check] [cluster] >>>>>>>>>>>>>>>>>>>>>>>>> : php.php.domain [Replicas] obsoleteReplicas : 0 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,715] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.AutoscalerRuleEvaluator} >>>>>>>>>>>>>>>>>>>>>>>>> - Minimum >>>>>>>>>>>>>>>>>>>>>>>>> check executed for : >>>>>>>>>>>>>>>>>>>>>>>>> org.apache.stratos.autoscaler.KubernetesClusterContext@21d906d0 >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,715] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesServiceClusterMonitor} >>>>>>>>>>>>>>>>>>>>>>>>> - >>>>>>>>>>>>>>>>>>>>>>>>> flag of rifReset : true flag of >>>>>>>>>>>>>>>>>>>>>>>>> memoryConsumptionReset : true flag of >>>>>>>>>>>>>>>>>>>>>>>>> loadAverageReset : true >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,715] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesServiceClusterMonitor} >>>>>>>>>>>>>>>>>>>>>>>>> - >>>>>>>>>>>>>>>>>>>>>>>>> Running scale check for [kub-cluster] : KubGrp1 >>>>>>>>>>>>>>>>>>>>>>>>> [cluster] : php.php.domain >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,716] INFO >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - >>>>>>>>>>>>>>>>>>>>>>>>> Running scaling rule >>>>>>>>>>>>>>>>>>>>>>>>> [kub-cluster] : KubGrp1 [cluster] : php.php.domain >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,716] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleTasksDelegator} >>>>>>>>>>>>>>>>>>>>>>>>> - Predicting the >>>>>>>>>>>>>>>>>>>>>>>>> value, [average]: 0.0 , [gradient]: 0.0 , [second >>>>>>>>>>>>>>>>>>>>>>>>> derivative]: 0.0 , [time >>>>>>>>>>>>>>>>>>>>>>>>> intervals]: 1 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,716] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleTasksDelegator} >>>>>>>>>>>>>>>>>>>>>>>>> - Predicting the >>>>>>>>>>>>>>>>>>>>>>>>> value, [average]: 28.8 , [gradient]: 0.0 , [second >>>>>>>>>>>>>>>>>>>>>>>>> derivative]: 0.0 , [time >>>>>>>>>>>>>>>>>>>>>>>>> intervals]: 1 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,717] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleTasksDelegator} >>>>>>>>>>>>>>>>>>>>>>>>> - Predicting the >>>>>>>>>>>>>>>>>>>>>>>>> value, [average]: 39.0 , [gradient]: -11000.0 , >>>>>>>>>>>>>>>>>>>>>>>>> [second derivative]: >>>>>>>>>>>>>>>>>>>>>>>>> 2000000.0 , [time intervals]: 1 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,717] INFO >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - >>>>>>>>>>>>>>>>>>>>>>>>> [scaling] [cluster] : >>>>>>>>>>>>>>>>>>>>>>>>> php.php.domain [Replicas] minReplicas : 3 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,717] INFO >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - >>>>>>>>>>>>>>>>>>>>>>>>> [scaling] [cluster] : >>>>>>>>>>>>>>>>>>>>>>>>> php.php.domain [Replicas] maxReplicas : 10 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718] INFO >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - >>>>>>>>>>>>>>>>>>>>>>>>> [scaling] [cluster] : >>>>>>>>>>>>>>>>>>>>>>>>> php.php.domain [Replicas] nonTerminated : 3 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718] INFO >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - >>>>>>>>>>>>>>>>>>>>>>>>> [scaling] [cluster] : >>>>>>>>>>>>>>>>>>>>>>>>> php.php.domain [Replicas] activeReplicas : 3 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718] INFO >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - >>>>>>>>>>>>>>>>>>>>>>>>> [scaling] [cluster] : >>>>>>>>>>>>>>>>>>>>>>>>> php.php.domain [RequestInFlight] predicted value : 0.0 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718] INFO >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - >>>>>>>>>>>>>>>>>>>>>>>>> [scaling] [cluster] : >>>>>>>>>>>>>>>>>>>>>>>>> php.php.domain [RequestInFlight] upper limit : 80.0 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718] INFO >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - >>>>>>>>>>>>>>>>>>>>>>>>> [scaling] [cluster] : >>>>>>>>>>>>>>>>>>>>>>>>> php.php.domain [RequestInFlight] lower limit : 5.0 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718] INFO >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - >>>>>>>>>>>>>>>>>>>>>>>>> [scaling] [cluster] : >>>>>>>>>>>>>>>>>>>>>>>>> php.php.domain [MemoryConsumption] predicted value : >>>>>>>>>>>>>>>>>>>>>>>>> 28.799999237060547 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718] INFO >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - >>>>>>>>>>>>>>>>>>>>>>>>> [scaling] [cluster] : >>>>>>>>>>>>>>>>>>>>>>>>> php.php.domain [MemoryConsumption] upper limit : 80.0 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718] INFO >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - >>>>>>>>>>>>>>>>>>>>>>>>> [scaling] [cluster] : >>>>>>>>>>>>>>>>>>>>>>>>> php.php.domain [MemoryConsumption] lower limit : 15.0 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718] INFO >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - >>>>>>>>>>>>>>>>>>>>>>>>> [scaling] [cluster] : >>>>>>>>>>>>>>>>>>>>>>>>> php.php.domain [LoadAverage] predicted value : >>>>>>>>>>>>>>>>>>>>>>>>> 989039.0 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,719] INFO >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - >>>>>>>>>>>>>>>>>>>>>>>>> [scaling] [cluster] : >>>>>>>>>>>>>>>>>>>>>>>>> php.php.domain [LoadAverage] upper limit : 180.0 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,719] INFO >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - >>>>>>>>>>>>>>>>>>>>>>>>> [scaling] [cluster] : >>>>>>>>>>>>>>>>>>>>>>>>> php.php.domain [LoadAverage] lower limit : 20.0 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,719] INFO >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - >>>>>>>>>>>>>>>>>>>>>>>>> [scaling] [cluster] : >>>>>>>>>>>>>>>>>>>>>>>>> php.php.domain scale-up action : true >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,719] INFO >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - >>>>>>>>>>>>>>>>>>>>>>>>> [scaling] [cluster] : >>>>>>>>>>>>>>>>>>>>>>>>> php.php.domain scale-down action : false >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,719] INFO >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - >>>>>>>>>>>>>>>>>>>>>>>>> [scaling] [cluster] : >>>>>>>>>>>>>>>>>>>>>>>>> php.php.domain [LoadAverage] predicted replicas : >>>>>>>>>>>>>>>>>>>>>>>>> 16484 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,720] INFO >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - >>>>>>>>>>>>>>>>>>>>>>>>> [scaling] [cluster] : >>>>>>>>>>>>>>>>>>>>>>>>> php.php.domain predicted replicas > max replicas : >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,720] INFO >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - >>>>>>>>>>>>>>>>>>>>>>>>> [scaling] Decided to >>>>>>>>>>>>>>>>>>>>>>>>> scale-up : [cluster] : php.php.domain >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,720] INFO >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - >>>>>>>>>>>>>>>>>>>>>>>>> [scaling-up] [cluster] : >>>>>>>>>>>>>>>>>>>>>>>>> php.php.domain valid number of replicas to expand : 10 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,720] INFO >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.client.cloud.controller.CloudControllerClient} >>>>>>>>>>>>>>>>>>>>>>>>> - Updating kubernetes replication controller via >>>>>>>>>>>>>>>>>>>>>>>>> cloud controller: >>>>>>>>>>>>>>>>>>>>>>>>> [cluster] php.php.domain [replicas] 10 >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,734] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.cloud.controller.impl.CloudControllerServiceImpl} >>>>>>>>>>>>>>>>>>>>>>>>> - >>>>>>>>>>>>>>>>>>>>>>>>> CloudControllerServiceImpl:updateContainers for >>>>>>>>>>>>>>>>>>>>>>>>> cluster : php.php.domain >>>>>>>>>>>>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,758] DEBUG >>>>>>>>>>>>>>>>>>>>>>>>> {org.apache.stratos.cloud.controller.impl.CloudControllerServiceImpl} >>>>>>>>>>>>>>>>>>>>>>>>> - >>>>>>>>>>>>>>>>>>>>>>>>> Cloud Controller is delegating request to update a >>>>>>>>>>>>>>>>>>>>>>>>> replication controller >>>>>>>>>>>>>>>>>>>>>>>>> php.php.domain to Kubernetes layer. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> I couldn't investigate the agent side, since >>>>>>>>>>>>>>>>>>>>>>>>> containers are destroyed. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Any idea? >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Thanks. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>>>>>>> Rajkumar Rajaratnam >>>>>>>>>>>>>>>>>>>>>>>>> Software Engineer | WSO2, Inc. >>>>>>>>>>>>>>>>>>>>>>>>> Mobile +94777568639 | +94783498120 >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>>>>>>>>>>> Manula Chathurika Thantriwatte >>>>>>>>>>>>>>>>>>>>>>> Software Engineer >>>>>>>>>>>>>>>>>>>>>>> WSO2 Inc. : http://wso2.com >>>>>>>>>>>>>>>>>>>>>>> lean . enterprise . middleware >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> email : manu...@wso2.com / man...@apache.org >>>>>>>>>>>>>>>>>>>>>>> phone : +94 772492511 >>>>>>>>>>>>>>>>>>>>>>> blog : http://manulachathurika.blogspot.com/ >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>>>> Rajkumar Rajaratnam >>>>>>>>>>>>>>>>>>>>>> Software Engineer | WSO2, Inc. >>>>>>>>>>>>>>>>>>>>>> Mobile +94777568639 | +94783498120 >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>>> Rajkumar Rajaratnam >>>>>>>>>>>>>>>>>>>>> Software Engineer | WSO2, Inc. >>>>>>>>>>>>>>>>>>>>> Mobile +94777568639 | +94783498120 >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>>>>>>>> Manula Chathurika Thantriwatte >>>>>>>>>>>>>>>>>>>> Software Engineer >>>>>>>>>>>>>>>>>>>> WSO2 Inc. : http://wso2.com >>>>>>>>>>>>>>>>>>>> lean . enterprise . middleware >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> email : manu...@wso2.com / man...@apache.org >>>>>>>>>>>>>>>>>>>> phone : +94 772492511 >>>>>>>>>>>>>>>>>>>> blog : http://manulachathurika.blogspot.com/ >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>>>>>>> Manula Chathurika Thantriwatte >>>>>>>>>>>>>>>>>>> Software Engineer >>>>>>>>>>>>>>>>>>> WSO2 Inc. : http://wso2.com >>>>>>>>>>>>>>>>>>> lean . enterprise . middleware >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> email : manu...@wso2.com / man...@apache.org >>>>>>>>>>>>>>>>>>> phone : +94 772492511 >>>>>>>>>>>>>>>>>>> blog : http://manulachathurika.blogspot.com/ >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>> Rajkumar Rajaratnam >>>>>>>>>>>>>>>>>> Software Engineer | WSO2, Inc. >>>>>>>>>>>>>>>>>> Mobile +94777568639 | +94783498120 >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>>>>> Manula Chathurika Thantriwatte >>>>>>>>>>>>>>>>> Software Engineer >>>>>>>>>>>>>>>>> WSO2 Inc. : http://wso2.com >>>>>>>>>>>>>>>>> lean . enterprise . middleware >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> email : manu...@wso2.com / man...@apache.org >>>>>>>>>>>>>>>>> phone : +94 772492511 >>>>>>>>>>>>>>>>> blog : http://manulachathurika.blogspot.com/ >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>> Rajkumar Rajaratnam >>>>>>>>>>>>>>>> Software Engineer | WSO2, Inc. >>>>>>>>>>>>>>>> Mobile +94777568639 | +94783498120 >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>>> Manula Chathurika Thantriwatte >>>>>>>>>>>>>>> Software Engineer >>>>>>>>>>>>>>> WSO2 Inc. : http://wso2.com >>>>>>>>>>>>>>> lean . enterprise . middleware >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> email : manu...@wso2.com / man...@apache.org >>>>>>>>>>>>>>> phone : +94 772492511 >>>>>>>>>>>>>>> blog : http://manulachathurika.blogspot.com/ >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>> Best Regards, >>>>>>>>>>>>>>> Nirmal >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Nirmal Fernando. >>>>>>>>>>>>>>> PPMC Member & Committer of Apache Stratos, >>>>>>>>>>>>>>> Senior Software Engineer, WSO2 Inc. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Blog: http://nirmalfdo.blogspot.com/ >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> Imesh Gunaratne >>>>>>>>>>>>>> >>>>>>>>>>>>>> Technical Lead, WSO2 >>>>>>>>>>>>>> Committer & PMC Member, Apache Stratos >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> Raj >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> >>>>>>>>>>>> Gayan Gunarathne >>>>>>>>>>>> Technical Lead >>>>>>>>>>>> WSO2 Inc. (http://wso2.com) >>>>>>>>>>>> email : gay...@wso2.com | mobile : +94 766819985 >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Best Regards, >>>>>>>>>>> Nirmal >>>>>>>>>>> >>>>>>>>>>> Nirmal Fernando. >>>>>>>>>>> PPMC Member & Committer of Apache Stratos, >>>>>>>>>>> Senior Software Engineer, WSO2 Inc. >>>>>>>>>>> >>>>>>>>>>> Blog: http://nirmalfdo.blogspot.com/ >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Imesh Gunaratne >>>>>>>>>> >>>>>>>>>> Technical Lead, WSO2 >>>>>>>>>> Committer & PMC Member, Apache Stratos >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Raj >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Raj >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Akila Ravihansa Perera >>>>>>> Software Engineer, WSO2 >>>>>>> >>>>>>> Blog: http://ravihansa3000.blogspot.com >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Best Regards, >>>>>> Nirmal >>>>>> >>>>>> Nirmal Fernando. >>>>>> PPMC Member & Committer of Apache Stratos, >>>>>> Senior Software Engineer, WSO2 Inc. >>>>>> >>>>>> Blog: http://nirmalfdo.blogspot.com/ >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Akila Ravihansa Perera >>>>> Software Engineer, WSO2 >>>>> >>>>> Blog: http://ravihansa3000.blogspot.com >>>>> >>>> >>>> >>>> >>>> -- >>>> Best Regards, >>>> Nirmal >>>> >>>> Nirmal Fernando. >>>> PPMC Member & Committer of Apache Stratos, >>>> Senior Software Engineer, WSO2 Inc. >>>> >>>> Blog: http://nirmalfdo.blogspot.com/ >>>> >>> >>> >>> >>> -- >>> Imesh Gunaratne >>> >>> Technical Lead, WSO2 >>> Committer & PMC Member, Apache Stratos >>> >> >> >> >> -- >> -- >> Lahiru Sandaruwan >> Committer and PMC member, Apache Stratos, >> Senior Software Engineer, >> WSO2 Inc., http://wso2.com >> lean.enterprise.middleware >> >> email: lahi...@wso2.com blog: http://lahiruwrites.blogspot.com/ >> linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146 >> >> > > > -- > Best Regards, > Nirmal > > Nirmal Fernando. > PPMC Member & Committer of Apache Stratos, > Senior Software Engineer, WSO2 Inc. > > Blog: http://nirmalfdo.blogspot.com/ > -- -- Lahiru Sandaruwan Committer and PMC member, Apache Stratos, Senior Software Engineer, WSO2 Inc., http://wso2.com lean.enterprise.middleware email: lahi...@wso2.com blog: http://lahiruwrites.blogspot.com/ linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
