On Thu, Nov 6, 2014 at 9:22 AM, Rajkumar Rajaratnam <rajkum...@wso2.com> wrote:
> Hi, > > No Imesh. There are no difference between *taking the the difference and > dividing it by 1000* and *dividing it by 1000 and taking the difference*. > Raj: I agree, in this scenario it makes no difference, however when we are applying a fomular the unit conversion should happen prior to applying values. Still I see a problem with this correction, take the same example: Gradient: -999.9999999999998 Last val: 7.000000000000001 First val: 12.0 Time Gap: 5 t1: 1415213232152 t2: 1415213232157 *Manual calulation:* The value difference: 7 - 12 = -5 Time difference: 1415213232157 - 1415213232152 = 5 ms => 5/1000 = 0.005 sec *Gradient = - 5/0.005 = -1000* *According to the current fix:* long millisecondsForASecond = 1000; long tGap = t2 - t1 > millisecondsForASecond ? t2 - t1 : millisecondsForASecond; double gradient = 0.0; if (tGap > 0) { gradient = ((lastVal - firstVal) * millisecondsForASecond) / tGap; } t1: 1415213232152 t2: 1415213232157 firstVal: 12 lastVall: 7 Gradient = -5 * 1000 / 1000 = - 5 Nirmal: Can you please justify this? Thanks On Thu, Nov 6, 2014 at 11:31 AM, Nirmal Fernando <nirmal070...@gmail.com> wrote: > > > On Thu, Nov 6, 2014 at 6:05 AM, Gayan Gunarathne <gay...@wso2.com> wrote: > >> AFAIU this is not related with the gradient calculation formula that we >> used. But It is related with the how we calculate gradient when the >> denominator become a small number.IMO when the divisor (timeInSeconds) >> approaches to the small number we do need to normalize the result. >> >> IMO we do have max and min scale up and down values. So based on that we >> can calculate the gradient on that value range.Simply if the denominator >> approaches zero result approaches to infinity. >> >> I think that's what Nirmal doing here by making the lower threshold as >> "millisecondsForASecond" >> > > You got it! thanks. > >> >> >> long tGap = t2 - t1 > millisecondsForASecond ? t2 - t1 : >> millisecondsForASecond; >> double gradient = 0.0; >> if (tGap > 0) { >> gradient = ((lastVal - firstVal) * millisecondsForASecond) / >> tGap; >> } >> >> >> Thanks, >> Gayan >> >> On Thu, Nov 6, 2014 at 9:22 AM, Rajkumar Rajaratnam <rajkum...@wso2.com> >> wrote: >> >>> Hi, >>> >>> On Thu, Nov 6, 2014 at 9:03 AM, Imesh Gunaratne <im...@apache.org> >>> wrote: >>> >>>> Hi Nirmal, >>>> >>>> I do not think the milliseconds to seconds convertion is correct here. >>>> >>>> As I see we are taking the difference of two timestamp values and then >>>> dividing it by 1000. The corect way might be to first divide each value by >>>> 1000 and then take the difference. >>>> >>> >>> No Imesh. There are no difference between *taking the the difference >>> and dividing it by 1000* and *dividing it by 1000 and taking the >>> difference*. >>> >>> >>>> On the other hand we might not need to convert these values to seconds >>>> since we are taking a time difference and calculating a gradient. >>>> >>>> I did a quick test with the following sample: >>>> Gradient: -999.9999999999998 Last val: 7.000000000000001 First val: >>>> 12.0 Time Gap: 5 t1: 1415213232152 t2: 1415213232157 >>>> >>>> *According previous code:* >>>> long tGap = t2 - t1; >>>> double gradient = 0.0; >>>> if (tGap > 0) { >>>> gradient = ((lastVal - firstVal) * 1000) / tGap; >>>> } >>>> >>>> t1: 1415213232152 t2: 1415213232157 firstVal: 12 lastVall: 7 >>>> gradient: -1000.0 >>>> >>>> *According to your fix:* >>>> long millisecondsForASecond = 1000; >>>> long tGap = t2 - t1 > millisecondsForASecond ? t2 - t1 : >>>> millisecondsForASecond; >>>> double gradient = 0.0; >>>> if (tGap > 0) { >>>> gradient = ((lastVal - firstVal) * millisecondsForASecond) >>>> / tGap; >>>> } >>>> >>>> t1: 1415213232152 t2: 1415213232157 firstVal: 12 lastVall: 7 >>>> gradient: -5.0 >>>> >>>> *According to an online gradient calculator:* >>>> gradient: -1 >>>> >>>> http://www.calculator.net/slope-calculator.html?type=1&x11=1415213232152&y11=12&x12=1415213232157&y12=7&x=27&y=19 >>>> >>>> According to the online gradient calculator (assuming their calculation >>>> is correct), the calculation in your fix is not correct. I believe the >>>> logic should be simple as follows: >>>> >>>> long tGap = t2 - t1; >>>> double gradient = 0.0; >>>> if (tGap > 0) { >>>> gradient = ((lastVal - firstVal)) / tGap; >>>> } >>>> >>>> t1: 1415213232152 t2: 1415213232157 firstVal: 12 lastVall: 7 >>>> gradient: -1.0 >>>> >>> >>> And AFAIK, standard way to calculate gradient = (v1-v2)/timeInSeconds. >>> >>> Well, we can divide it by what ever time units we wants, but the >>> result's unit will be different. >>> >>> In Nirmal's case, he get will get *stats/seconds*. In your case, you >>> will get *stats/milliseconds* >>> >>> And AS prediction is based on *seconds*, CEP should use *seconds* for >>> the calculation. >>> >>> wdyt? >>> >>> Thanks. >>> >>> >>>> Thanks >>>> >>>> >>>> On Thu, Nov 6, 2014 at 1:32 AM, Nirmal Fernando <nirmal070...@gmail.com >>>> > wrote: >>>> >>>>> Hi Guys, >>>>> >>>>> So, I got a chance to reproduce and analyze this issue. >>>>> >>>>> How we calculate gradient of two events? >>>>> >>>>> Say the events are; e1(t1,v1) and e2(t2,v2) >>>>> >>>>> tx - time in *milliseconds* when xth event occurred >>>>> vx - value (memory, cpu etc.) that xth event carries >>>>> >>>>> *time gap = t(2-1) = t2 -t1 milliseconds* >>>>> time gap in seconds = t(2-1) = (t2 - t1)/1000 seconds >>>>> >>>>> Hence, >>>>> *Gradient = (v2 - v1) / t(2-1) = ( (v2 - v1) * 1000 ) / (t2 - t1)* >>>>> >>>>> I've enabled debug logs for CEP extension; >>>>> >>>>> log4j.logger.org.apache.stratos.cep.extension=DEBUG >>>>> >>>>> Please find the following 3 logs extracted from the debug logs; >>>>> >>>>> =================================================================== >>>>> TID: [0] [STRATOS] [2014-11-05 19:47:27,073] DEBUG >>>>> {org.apache.stratos.cep.extension.SecondDerivativeFinderWindowProcessor} - >>>>> Gradient: -0.1996007984031936 Last val: 9.0 First val: 12.0 *Time >>>>> Gap: 15030* t1: 1415213202095 t2: 1415213217125 hash: 155426542 >>>>> >>>>> TID: [0] [STRATOS] [2014-11-05 19:47:27,073] DEBUG >>>>> {org.apache.stratos.cep.extension.SecondDerivativeFinderWindowProcessor} >>>>> - Gradient: >>>>> -999.9999999999998 Last val: 7.000000000000001 First val: 12.0 *Time >>>>> Gap: 5* t1: 1415213232152 t2: 1415213232157 hash: 155426542 >>>>> >>>>> TID: [0] [STRATOS] [2014-11-05 19:47:27,074] DEBUG >>>>> {org.apache.stratos.cep.extension.SecondDerivativeFinderWindowProcessor} - >>>>> Gradient: -44.34884666437174 Last val: -999.9999999999998 First val: >>>>> -0.1996007984031936 *Time Gap: 22544* t1: 1415213209610 t2: >>>>> 1415213232154 hash: 155426542 >>>>> =================================================================== >>>>> >>>>> So, as you can see the reason behind a large value is when the time >>>>> gap between two subjected events is less than 1 second. This could happen >>>>> since events are coming from different asynchronous agents and also when >>>>> there are less number of events. >>>>> >>>>> So, the fix I propose is a very simple one and it will not compromise >>>>> anything AFAIS. >>>>> >>>>> Fix is to calculate time gap as follows; >>>>> *____ t2 - t1* >>>>> *| yes?* >>>>> *time gap = t(2-1) = t2 -t1 > 1000 ---* >>>>> *|____ 1000* >>>>> >>>>> I have tested this and works fine. >>>>> >>>>> >>>>> >>>>> ---------- Forwarded message ---------- >>>>> From: Manula Chathurika Thantriwatte <manu...@wso2.com> >>>>> Date: Tue, Oct 28, 2014 at 5:16 AM >>>>> Subject: Re: CEP sends very large values for gradient and second >>>>> derivative of load average >>>>> To: "dev@stratos.apache.org" <dev@stratos.apache.org> >>>>> >>>>> >>>>> Hi Raj, >>>>> >>>>> Yes it has. But it's very easy to find the values from the agent with >>>>> the time stamp. Then we can search that time stamp in the CEP trace logs >>>>> and compare the summarized values. >>>>> >>>>> Thanks ! >>>>> >>>>> On Mon, Oct 27, 2014 at 9:23 PM, Rajkumar Rajaratnam < >>>>> rajkum...@wso2.com> wrote: >>>>> >>>>>> >>>>>> >>>>>> On Mon, Oct 27, 2014 at 7:56 PM, Manula Chathurika Thantriwatte < >>>>>> manu...@wso2.com> wrote: >>>>>> >>>>>>> Hi Raj, >>>>>>> >>>>>>> Shall we enable the cartridge agent debug logs. Then we can compare >>>>>>> both CEP and agent logs with the time stamp. Then we can narrow down >>>>>>> where >>>>>>> the actual problem is. >>>>>>> >>>>>> >>>>>> Manula, cep-trace log have the values sent by cartridge agent also >>>>>> right? >>>>>> >>>>>> >>>>>>> >>>>>>> Thanks ! >>>>>>> >>>>>>> On Mon, Oct 27, 2014 at 5:06 PM, Rajkumar Rajaratnam < >>>>>>> rajkum...@wso2.com> wrote: >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Mon, Oct 27, 2014 at 12:53 PM, Manula Chathurika Thantriwatte < >>>>>>>> manu...@wso2.com> wrote: >>>>>>>> >>>>>>>>> Hi Raj, >>>>>>>>> >>>>>>>>> Is this happen frequently or intermittently ? Once I go though the >>>>>>>>> logs I'm unable to find the high input values for load average. But >>>>>>>>> output >>>>>>>>> have high values. >>>>>>>>> >>>>>>>>> 22:10:38,698 [-] [Siddhi-Scheduler-pool-10-thread-1] INFO >>>>>>>>> EVENT_TRACE_LOGGER TenantId=-1234 : Output Event Adaptor : >>>>>>>>> JMSOutputAdaptor, sent >>>>>>>>> >>>>>>>>> {"org.apache.stratos.messaging.event.health.stat.SecondDerivativeOfLoadAverageEvent":{"message":{"clusterId":"php.php.domain","networkPartitionId":"","value":"2000000.0"}}} >>>>>>>>> >>>>>>>> >>>>>>>> This is happening intermittently. Experienced now too. This is a >>>>>>>> serious issue. We need to somehow sort this out. >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> Also networkPartitionId is empty. In the current execution plans >>>>>>>>> it took networkPartitionId for the calculation. IMO we need to update >>>>>>>>> the >>>>>>>>> CEP execution plans for 4.1.0. >>>>>>>>> >>>>>>>>> Thanks ! >>>>>>>>> >>>>>>>>> >>>>>>>>> On Sun, Oct 26, 2014 at 8:49 AM, Manula Chathurika Thantriwatte < >>>>>>>>> manu...@wso2.com> wrote: >>>>>>>>> >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> Thanks Raj. I'll go through them. >>>>>>>>>> >>>>>>>>>> Thanks ! >>>>>>>>>> >>>>>>>>>> On Sun, Oct 26, 2014 at 8:33 AM, Rajkumar Rajaratnam < >>>>>>>>>> rajkum...@wso2.com> wrote: >>>>>>>>>> >>>>>>>>>>> Hi Manula, >>>>>>>>>>> >>>>>>>>>>> You can see those values in the same log I attached in the >>>>>>>>>>> previous mail. >>>>>>>>>>> >>>>>>>>>>> Thanks. >>>>>>>>>>> >>>>>>>>>>> On Sun, Oct 26, 2014 at 8:29 AM, Rajkumar Rajaratnam < >>>>>>>>>>> rajkum...@wso2.com> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi Chamila, >>>>>>>>>>>> >>>>>>>>>>>> I couldn't find any large values sent by agent. There are >>>>>>>>>>>> traces for CEP sending large values. Find the cep trace log here >>>>>>>>>>>> [1] >>>>>>>>>>>> >>>>>>>>>>>> 1. >>>>>>>>>>>> https://drive.google.com/file/d/0B1haIleqJMHIS3FFVFpfa0JlbVE/view?usp=sharing >>>>>>>>>>>> >>>>>>>>>>>> Thanks. >>>>>>>>>>>> >>>>>>>>>>>> On Sun, Oct 26, 2014 at 8:23 AM, Manula Chathurika Thantriwatte >>>>>>>>>>>> <manu...@wso2.com> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi, >>>>>>>>>>>>> >>>>>>>>>>>>> Shall we enable the cartridge agent debug logs and see what >>>>>>>>>>>>> are the load average values sent from the agent. If those values >>>>>>>>>>>>> are normal >>>>>>>>>>>>> then we can narrow down to the CEP. >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks ! >>>>>>>>>>>>> >>>>>>>>>>>>> On Sat, Oct 25, 2014 at 10:43 PM, Chamila De Alwis < >>>>>>>>>>>>> chami...@wso2.com> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hi Raj, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Can you inspect the CEP trace logs for the same time period? >>>>>>>>>>>>>> It will have the events published from the agent. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>> Chamila de Alwis >>>>>>>>>>>>>> Software Engineer | WSO2 | +94772207163 >>>>>>>>>>>>>> Blog: code.chamiladealwis.com >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Sat, Oct 25, 2014 at 10:39 PM, Rajkumar Rajaratnam < >>>>>>>>>>>>>> rajkum...@wso2.com> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I am doing a test round with M3 with python agent. And faced >>>>>>>>>>>>>>> this very strange scenario, $Subject. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,693] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - >>>>>>>>>>>>>>> Load average >>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,693] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>> - Grad of >>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] -2000.0 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,694] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - >>>>>>>>>>>>>>> Load average >>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,694] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>> - Grad of >>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 0.0 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,694] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - >>>>>>>>>>>>>>> Load average >>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,694] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>> - Avg >>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 1.9090909 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,694] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - >>>>>>>>>>>>>>> Load average >>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,694] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>> - Avg >>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value] >>>>>>>>>>>>>>> 28.718182 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,695] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>> - Second >>>>>>>>>>>>>>> Derivation of Memory Consumption event: [cluster] >>>>>>>>>>>>>>> php.php.domain [value] >>>>>>>>>>>>>>> 200000.0 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,695] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>> - Grad of >>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 0.0 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,695] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - >>>>>>>>>>>>>>> Load average >>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,695] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>> - Second >>>>>>>>>>>>>>> Derivation of load avg event: [cluster] php.php.domain [value] >>>>>>>>>>>>>>> 0.0 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,695] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - >>>>>>>>>>>>>>> Load average >>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,696] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>> - Second >>>>>>>>>>>>>>> Derivation of load avg event: [cluster] php.php.domain [value] >>>>>>>>>>>>>>> 0.0 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,696] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - >>>>>>>>>>>>>>> Load average >>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,696] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>> - Avg >>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 3.3636363 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,696] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - >>>>>>>>>>>>>>> Load average >>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,696] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>> - Grad of >>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value] >>>>>>>>>>>>>>> 0.11670045 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,696] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - >>>>>>>>>>>>>>> Memory >>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub >>>>>>>>>>>>>>> cluster] KubGrp1 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>> - Avg >>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value] 28.7 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - >>>>>>>>>>>>>>> Memory >>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub >>>>>>>>>>>>>>> cluster] KubGrp1 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>> - Second >>>>>>>>>>>>>>> Derivation of Memory Consumption event: [cluster] >>>>>>>>>>>>>>> php.php.domain [value] 0.0 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - >>>>>>>>>>>>>>> Memory >>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub >>>>>>>>>>>>>>> cluster] KubGrp1 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>> - Avg >>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value] 28.7 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - >>>>>>>>>>>>>>> Memory >>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub >>>>>>>>>>>>>>> cluster] KubGrp1 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>> - Avg >>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 5.0 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - >>>>>>>>>>>>>>> Load average >>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>> - Grad of >>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 2000.0 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,698] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - >>>>>>>>>>>>>>> Load average >>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,698] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>> - Avg >>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value] 28.7 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,698] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - >>>>>>>>>>>>>>> Memory >>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub >>>>>>>>>>>>>>> cluster] KubGrp1 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,698] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>> - Grad of >>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value] 0.0 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,698] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - >>>>>>>>>>>>>>> Memory >>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub >>>>>>>>>>>>>>> cluster] KubGrp1 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,699] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>> - Second >>>>>>>>>>>>>>> Derivation of Memory Consumption event: [cluster] >>>>>>>>>>>>>>> php.php.domain [value] 0.0 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,699] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - >>>>>>>>>>>>>>> Memory >>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub >>>>>>>>>>>>>>> cluster] KubGrp1 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,700] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>> - Second >>>>>>>>>>>>>>> Derivation of load avg event: [cluster] php.php.domain [value] >>>>>>>>>>>>>>> 2000000.0 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,700] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - >>>>>>>>>>>>>>> Load average >>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,700] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>> - Grad of >>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 13500.0 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,700] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - >>>>>>>>>>>>>>> Load average >>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,701] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>> - Grad of >>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value] 50.0 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,701] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - >>>>>>>>>>>>>>> Memory >>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub >>>>>>>>>>>>>>> cluster] KubGrp1 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,701] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>> - Avg >>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value] >>>>>>>>>>>>>>> 28.738462 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,701] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - >>>>>>>>>>>>>>> Memory >>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub >>>>>>>>>>>>>>> cluster] KubGrp1 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,702] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>> - Avg >>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 2.3333333 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,702] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - >>>>>>>>>>>>>>> Load average >>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,702] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>> - Grad of >>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 0.0 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,702] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - >>>>>>>>>>>>>>> Load average >>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,703] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>> - Grad of >>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value] 0.0 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,703] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - >>>>>>>>>>>>>>> Memory >>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub >>>>>>>>>>>>>>> cluster] KubGrp1 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,703] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>> - Avg >>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 31.0 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,703] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - >>>>>>>>>>>>>>> Load average >>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,704] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>> - Grad of >>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value] 0.0 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,704] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - >>>>>>>>>>>>>>> Memory >>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub >>>>>>>>>>>>>>> cluster] KubGrp1 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,704] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>> - Avg >>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 39.0 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,704] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - >>>>>>>>>>>>>>> Load average >>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,704] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>> - Avg >>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 39.0 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,705] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - >>>>>>>>>>>>>>> Load average >>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,706] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>> - Grad of >>>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] -11000.0 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,706] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - >>>>>>>>>>>>>>> Load average >>>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,708] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>> - Grad of >>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value] 0.0 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,708] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - >>>>>>>>>>>>>>> Memory >>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub >>>>>>>>>>>>>>> cluster] KubGrp1 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,709] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>>> - Avg >>>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value] 28.8 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,709] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - >>>>>>>>>>>>>>> Memory >>>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub >>>>>>>>>>>>>>> cluster] KubGrp1 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,811] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.cloud.controller.util.PodActivationWatcher} >>>>>>>>>>>>>>> - >>>>>>>>>>>>>>> PodActivationWatcher running : Running >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,942] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.cloud.controller.util.PodActivationWatcher} >>>>>>>>>>>>>>> - >>>>>>>>>>>>>>> PodActivationWatcher running : Running >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,945] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.cloud.controller.util.PodActivationWatcher} >>>>>>>>>>>>>>> - >>>>>>>>>>>>>>> PodActivationWatcher running : Running >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,712] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesServiceClusterMonitor} >>>>>>>>>>>>>>> - >>>>>>>>>>>>>>> KubernetesServiceClusterMonitor is running.. >>>>>>>>>>>>>>> KubernetesServiceClusterMonitor [ >>>>>>>>>>>>>>> kubernetesHostClusterId=KubGrp1, >>>>>>>>>>>>>>> clusterId=php.php.domain, serviceId=php] >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,713] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesServiceClusterMonitor} >>>>>>>>>>>>>>> - >>>>>>>>>>>>>>> Running min check for [kub-cluster] : KubGrp1 [cluster] : >>>>>>>>>>>>>>> php.php.domain >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,713] INFO >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - Running minimum >>>>>>>>>>>>>>> rule: >>>>>>>>>>>>>>> [kub-cluster] KubGrp1 [cluster] php.php.domain >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,714] INFO >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [min-check] >>>>>>>>>>>>>>> [cluster] : >>>>>>>>>>>>>>> php.php.domain [Replicas] nonTerminated : 3 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,714] INFO >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [min-check] >>>>>>>>>>>>>>> [cluster] : >>>>>>>>>>>>>>> php.php.domain [Replicas] minReplicas : 3 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,715] INFO >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - Running >>>>>>>>>>>>>>> obsolete containers >>>>>>>>>>>>>>> rule [kub-cluster] : KubGrp1 [cluster] : php.php.domain >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,715] INFO >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - >>>>>>>>>>>>>>> [obsolete-check] [cluster] >>>>>>>>>>>>>>> : php.php.domain [Replicas] obsoleteReplicas : 0 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,715] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.AutoscalerRuleEvaluator} - >>>>>>>>>>>>>>> Minimum >>>>>>>>>>>>>>> check executed for : >>>>>>>>>>>>>>> org.apache.stratos.autoscaler.KubernetesClusterContext@21d906d0 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,715] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesServiceClusterMonitor} >>>>>>>>>>>>>>> - >>>>>>>>>>>>>>> flag of rifReset : true flag of memoryConsumptionReset : true >>>>>>>>>>>>>>> flag of >>>>>>>>>>>>>>> loadAverageReset : true >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,715] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesServiceClusterMonitor} >>>>>>>>>>>>>>> - >>>>>>>>>>>>>>> Running scale check for [kub-cluster] : KubGrp1 [cluster] : >>>>>>>>>>>>>>> php.php.domain >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,716] INFO >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - Running scaling >>>>>>>>>>>>>>> rule >>>>>>>>>>>>>>> [kub-cluster] : KubGrp1 [cluster] : php.php.domain >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,716] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleTasksDelegator} - >>>>>>>>>>>>>>> Predicting the >>>>>>>>>>>>>>> value, [average]: 0.0 , [gradient]: 0.0 , [second derivative]: >>>>>>>>>>>>>>> 0.0 , [time >>>>>>>>>>>>>>> intervals]: 1 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,716] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleTasksDelegator} - >>>>>>>>>>>>>>> Predicting the >>>>>>>>>>>>>>> value, [average]: 28.8 , [gradient]: 0.0 , [second derivative]: >>>>>>>>>>>>>>> 0.0 , [time >>>>>>>>>>>>>>> intervals]: 1 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,717] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleTasksDelegator} - >>>>>>>>>>>>>>> Predicting the >>>>>>>>>>>>>>> value, [average]: 39.0 , [gradient]: -11000.0 , [second >>>>>>>>>>>>>>> derivative]: >>>>>>>>>>>>>>> 2000000.0 , [time intervals]: 1 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,717] INFO >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling] >>>>>>>>>>>>>>> [cluster] : >>>>>>>>>>>>>>> php.php.domain [Replicas] minReplicas : 3 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,717] INFO >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling] >>>>>>>>>>>>>>> [cluster] : >>>>>>>>>>>>>>> php.php.domain [Replicas] maxReplicas : 10 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718] INFO >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling] >>>>>>>>>>>>>>> [cluster] : >>>>>>>>>>>>>>> php.php.domain [Replicas] nonTerminated : 3 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718] INFO >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling] >>>>>>>>>>>>>>> [cluster] : >>>>>>>>>>>>>>> php.php.domain [Replicas] activeReplicas : 3 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718] INFO >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling] >>>>>>>>>>>>>>> [cluster] : >>>>>>>>>>>>>>> php.php.domain [RequestInFlight] predicted value : 0.0 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718] INFO >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling] >>>>>>>>>>>>>>> [cluster] : >>>>>>>>>>>>>>> php.php.domain [RequestInFlight] upper limit : 80.0 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718] INFO >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling] >>>>>>>>>>>>>>> [cluster] : >>>>>>>>>>>>>>> php.php.domain [RequestInFlight] lower limit : 5.0 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718] INFO >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling] >>>>>>>>>>>>>>> [cluster] : >>>>>>>>>>>>>>> php.php.domain [MemoryConsumption] predicted value : >>>>>>>>>>>>>>> 28.799999237060547 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718] INFO >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling] >>>>>>>>>>>>>>> [cluster] : >>>>>>>>>>>>>>> php.php.domain [MemoryConsumption] upper limit : 80.0 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718] INFO >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling] >>>>>>>>>>>>>>> [cluster] : >>>>>>>>>>>>>>> php.php.domain [MemoryConsumption] lower limit : 15.0 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718] INFO >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling] >>>>>>>>>>>>>>> [cluster] : >>>>>>>>>>>>>>> php.php.domain [LoadAverage] predicted value : 989039.0 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,719] INFO >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling] >>>>>>>>>>>>>>> [cluster] : >>>>>>>>>>>>>>> php.php.domain [LoadAverage] upper limit : 180.0 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,719] INFO >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling] >>>>>>>>>>>>>>> [cluster] : >>>>>>>>>>>>>>> php.php.domain [LoadAverage] lower limit : 20.0 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,719] INFO >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling] >>>>>>>>>>>>>>> [cluster] : >>>>>>>>>>>>>>> php.php.domain scale-up action : true >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,719] INFO >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling] >>>>>>>>>>>>>>> [cluster] : >>>>>>>>>>>>>>> php.php.domain scale-down action : false >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,719] INFO >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling] >>>>>>>>>>>>>>> [cluster] : >>>>>>>>>>>>>>> php.php.domain [LoadAverage] predicted replicas : 16484 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,720] INFO >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling] >>>>>>>>>>>>>>> [cluster] : >>>>>>>>>>>>>>> php.php.domain predicted replicas > max replicas : >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,720] INFO >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling] >>>>>>>>>>>>>>> Decided to >>>>>>>>>>>>>>> scale-up : [cluster] : php.php.domain >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,720] INFO >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling-up] >>>>>>>>>>>>>>> [cluster] : >>>>>>>>>>>>>>> php.php.domain valid number of replicas to expand : 10 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,720] INFO >>>>>>>>>>>>>>> {org.apache.stratos.autoscaler.client.cloud.controller.CloudControllerClient} >>>>>>>>>>>>>>> - Updating kubernetes replication controller via cloud >>>>>>>>>>>>>>> controller: >>>>>>>>>>>>>>> [cluster] php.php.domain [replicas] 10 >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,734] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.cloud.controller.impl.CloudControllerServiceImpl} >>>>>>>>>>>>>>> - >>>>>>>>>>>>>>> CloudControllerServiceImpl:updateContainers for cluster : >>>>>>>>>>>>>>> php.php.domain >>>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,758] DEBUG >>>>>>>>>>>>>>> {org.apache.stratos.cloud.controller.impl.CloudControllerServiceImpl} >>>>>>>>>>>>>>> - >>>>>>>>>>>>>>> Cloud Controller is delegating request to update a replication >>>>>>>>>>>>>>> controller >>>>>>>>>>>>>>> php.php.domain to Kubernetes layer. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I couldn't investigate the agent side, since containers are >>>>>>>>>>>>>>> destroyed. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Any idea? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>> Rajkumar Rajaratnam >>>>>>>>>>>>>>> Software Engineer | WSO2, Inc. >>>>>>>>>>>>>>> Mobile +94777568639 | +94783498120 >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> Regards, >>>>>>>>>>>>> Manula Chathurika Thantriwatte >>>>>>>>>>>>> Software Engineer >>>>>>>>>>>>> WSO2 Inc. : http://wso2.com >>>>>>>>>>>>> lean . enterprise . middleware >>>>>>>>>>>>> >>>>>>>>>>>>> email : manu...@wso2.com / man...@apache.org >>>>>>>>>>>>> phone : +94 772492511 >>>>>>>>>>>>> blog : http://manulachathurika.blogspot.com/ >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> Rajkumar Rajaratnam >>>>>>>>>>>> Software Engineer | WSO2, Inc. >>>>>>>>>>>> Mobile +94777568639 | +94783498120 >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Rajkumar Rajaratnam >>>>>>>>>>> Software Engineer | WSO2, Inc. >>>>>>>>>>> Mobile +94777568639 | +94783498120 >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Regards, >>>>>>>>>> Manula Chathurika Thantriwatte >>>>>>>>>> Software Engineer >>>>>>>>>> WSO2 Inc. : http://wso2.com >>>>>>>>>> lean . enterprise . middleware >>>>>>>>>> >>>>>>>>>> email : manu...@wso2.com / man...@apache.org >>>>>>>>>> phone : +94 772492511 >>>>>>>>>> blog : http://manulachathurika.blogspot.com/ >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Regards, >>>>>>>>> Manula Chathurika Thantriwatte >>>>>>>>> Software Engineer >>>>>>>>> WSO2 Inc. : http://wso2.com >>>>>>>>> lean . enterprise . middleware >>>>>>>>> >>>>>>>>> email : manu...@wso2.com / man...@apache.org >>>>>>>>> phone : +94 772492511 >>>>>>>>> blog : http://manulachathurika.blogspot.com/ >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Rajkumar Rajaratnam >>>>>>>> Software Engineer | WSO2, Inc. >>>>>>>> Mobile +94777568639 | +94783498120 >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Regards, >>>>>>> Manula Chathurika Thantriwatte >>>>>>> Software Engineer >>>>>>> WSO2 Inc. : http://wso2.com >>>>>>> lean . enterprise . middleware >>>>>>> >>>>>>> email : manu...@wso2.com / man...@apache.org >>>>>>> phone : +94 772492511 >>>>>>> blog : http://manulachathurika.blogspot.com/ >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Rajkumar Rajaratnam >>>>>> Software Engineer | WSO2, Inc. >>>>>> Mobile +94777568639 | +94783498120 >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Regards, >>>>> Manula Chathurika Thantriwatte >>>>> Software Engineer >>>>> WSO2 Inc. : http://wso2.com >>>>> lean . enterprise . middleware >>>>> >>>>> email : manu...@wso2.com / man...@apache.org >>>>> phone : +94 772492511 >>>>> blog : http://manulachathurika.blogspot.com/ >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Best Regards, >>>>> Nirmal >>>>> >>>>> Nirmal Fernando. >>>>> PPMC Member & Committer of Apache Stratos, >>>>> Senior Software Engineer, WSO2 Inc. >>>>> >>>>> Blog: http://nirmalfdo.blogspot.com/ >>>>> >>>> >>>> >>>> >>>> -- >>>> Imesh Gunaratne >>>> >>>> Technical Lead, WSO2 >>>> Committer & PMC Member, Apache Stratos >>>> >>> >>> >>> >>> -- >>> Raj >>> >> >> >> >> -- >> >> Gayan Gunarathne >> Technical Lead >> WSO2 Inc. (http://wso2.com) >> email : gay...@wso2.com | mobile : +94 766819985 >> >> > > > > -- > Best Regards, > Nirmal > > Nirmal Fernando. > PPMC Member & Committer of Apache Stratos, > Senior Software Engineer, WSO2 Inc. > > Blog: http://nirmalfdo.blogspot.com/ > -- Imesh Gunaratne Technical Lead, WSO2 Committer & PMC Member, Apache Stratos