On Thu, Nov 6, 2014 at 6:05 AM, Gayan Gunarathne <gay...@wso2.com> wrote:
> AFAIU this is not related with the gradient calculation formula that we > used. But It is related with the how we calculate gradient when the > denominator become a small number.IMO when the divisor (timeInSeconds) > approaches to the small number we do need to normalize the result. > > IMO we do have max and min scale up and down values. So based on that we > can calculate the gradient on that value range.Simply if the denominator > approaches zero result approaches to infinity. > > I think that's what Nirmal doing here by making the lower threshold as > "millisecondsForASecond" > You got it! thanks. > > > long tGap = t2 - t1 > millisecondsForASecond ? t2 - t1 : > millisecondsForASecond; > double gradient = 0.0; > if (tGap > 0) { > gradient = ((lastVal - firstVal) * millisecondsForASecond) / > tGap; > } > > > Thanks, > Gayan > > On Thu, Nov 6, 2014 at 9:22 AM, Rajkumar Rajaratnam <rajkum...@wso2.com> > wrote: > >> Hi, >> >> On Thu, Nov 6, 2014 at 9:03 AM, Imesh Gunaratne <im...@apache.org> wrote: >> >>> Hi Nirmal, >>> >>> I do not think the milliseconds to seconds convertion is correct here. >>> >>> As I see we are taking the difference of two timestamp values and then >>> dividing it by 1000. The corect way might be to first divide each value by >>> 1000 and then take the difference. >>> >> >> No Imesh. There are no difference between *taking the the difference and >> dividing it by 1000* and *dividing it by 1000 and taking the difference*. >> >> >> >>> On the other hand we might not need to convert these values to seconds >>> since we are taking a time difference and calculating a gradient. >>> >>> I did a quick test with the following sample: >>> Gradient: -999.9999999999998 Last val: 7.000000000000001 First val: 12.0 >>> Time Gap: 5 t1: 1415213232152 t2: 1415213232157 >>> >>> *According previous code:* >>> long tGap = t2 - t1; >>> double gradient = 0.0; >>> if (tGap > 0) { >>> gradient = ((lastVal - firstVal) * 1000) / tGap; >>> } >>> >>> t1: 1415213232152 t2: 1415213232157 firstVal: 12 lastVall: 7 >>> gradient: -1000.0 >>> >>> *According to your fix:* >>> long millisecondsForASecond = 1000; >>> long tGap = t2 - t1 > millisecondsForASecond ? t2 - t1 : >>> millisecondsForASecond; >>> double gradient = 0.0; >>> if (tGap > 0) { >>> gradient = ((lastVal - firstVal) * millisecondsForASecond) / >>> tGap; >>> } >>> >>> t1: 1415213232152 t2: 1415213232157 firstVal: 12 lastVall: 7 >>> gradient: -5.0 >>> >>> *According to an online gradient calculator:* >>> gradient: -1 >>> >>> http://www.calculator.net/slope-calculator.html?type=1&x11=1415213232152&y11=12&x12=1415213232157&y12=7&x=27&y=19 >>> >>> According to the online gradient calculator (assuming their calculation >>> is correct), the calculation in your fix is not correct. I believe the >>> logic should be simple as follows: >>> >>> long tGap = t2 - t1; >>> double gradient = 0.0; >>> if (tGap > 0) { >>> gradient = ((lastVal - firstVal)) / tGap; >>> } >>> >>> t1: 1415213232152 t2: 1415213232157 firstVal: 12 lastVall: 7 >>> gradient: -1.0 >>> >> >> And AFAIK, standard way to calculate gradient = (v1-v2)/timeInSeconds. >> >> Well, we can divide it by what ever time units we wants, but the result's >> unit will be different. >> >> In Nirmal's case, he get will get *stats/seconds*. In your case, you >> will get *stats/milliseconds* >> >> And AS prediction is based on *seconds*, CEP should use *seconds* for >> the calculation. >> >> wdyt? >> >> Thanks. >> >> >>> Thanks >>> >>> >>> On Thu, Nov 6, 2014 at 1:32 AM, Nirmal Fernando <nirmal070...@gmail.com> >>> wrote: >>> >>>> Hi Guys, >>>> >>>> So, I got a chance to reproduce and analyze this issue. >>>> >>>> How we calculate gradient of two events? >>>> >>>> Say the events are; e1(t1,v1) and e2(t2,v2) >>>> >>>> tx - time in *milliseconds* when xth event occurred >>>> vx - value (memory, cpu etc.) that xth event carries >>>> >>>> *time gap = t(2-1) = t2 -t1 milliseconds* >>>> time gap in seconds = t(2-1) = (t2 - t1)/1000 seconds >>>> >>>> Hence, >>>> *Gradient = (v2 - v1) / t(2-1) = ( (v2 - v1) * 1000 ) / (t2 - t1)* >>>> >>>> I've enabled debug logs for CEP extension; >>>> >>>> log4j.logger.org.apache.stratos.cep.extension=DEBUG >>>> >>>> Please find the following 3 logs extracted from the debug logs; >>>> >>>> =================================================================== >>>> TID: [0] [STRATOS] [2014-11-05 19:47:27,073] DEBUG >>>> {org.apache.stratos.cep.extension.SecondDerivativeFinderWindowProcessor} - >>>> Gradient: -0.1996007984031936 Last val: 9.0 First val: 12.0 *Time >>>> Gap: 15030* t1: 1415213202095 t2: 1415213217125 hash: 155426542 >>>> >>>> TID: [0] [STRATOS] [2014-11-05 19:47:27,073] DEBUG >>>> {org.apache.stratos.cep.extension.SecondDerivativeFinderWindowProcessor} - >>>> Gradient: >>>> -999.9999999999998 Last val: 7.000000000000001 First val: 12.0 *Time >>>> Gap: 5* t1: 1415213232152 t2: 1415213232157 hash: 155426542 >>>> >>>> TID: [0] [STRATOS] [2014-11-05 19:47:27,074] DEBUG >>>> {org.apache.stratos.cep.extension.SecondDerivativeFinderWindowProcessor} - >>>> Gradient: -44.34884666437174 Last val: -999.9999999999998 First val: >>>> -0.1996007984031936 *Time Gap: 22544* t1: 1415213209610 t2: >>>> 1415213232154 hash: 155426542 >>>> =================================================================== >>>> >>>> So, as you can see the reason behind a large value is when the time gap >>>> between two subjected events is less than 1 second. This could happen since >>>> events are coming from different asynchronous agents and also when there >>>> are less number of events. >>>> >>>> So, the fix I propose is a very simple one and it will not compromise >>>> anything AFAIS. >>>> >>>> Fix is to calculate time gap as follows; >>>> *____ t2 - t1* >>>> *| yes?* >>>> *time gap = t(2-1) = t2 -t1 > 1000 ---* >>>> *|____ 1000* >>>> >>>> I have tested this and works fine. >>>> >>>> >>>> >>>> ---------- Forwarded message ---------- >>>> From: Manula Chathurika Thantriwatte <manu...@wso2.com> >>>> Date: Tue, Oct 28, 2014 at 5:16 AM >>>> Subject: Re: CEP sends very large values for gradient and second >>>> derivative of load average >>>> To: "dev@stratos.apache.org" <dev@stratos.apache.org> >>>> >>>> >>>> Hi Raj, >>>> >>>> Yes it has. But it's very easy to find the values from the agent with >>>> the time stamp. Then we can search that time stamp in the CEP trace logs >>>> and compare the summarized values. >>>> >>>> Thanks ! >>>> >>>> On Mon, Oct 27, 2014 at 9:23 PM, Rajkumar Rajaratnam < >>>> rajkum...@wso2.com> wrote: >>>> >>>>> >>>>> >>>>> On Mon, Oct 27, 2014 at 7:56 PM, Manula Chathurika Thantriwatte < >>>>> manu...@wso2.com> wrote: >>>>> >>>>>> Hi Raj, >>>>>> >>>>>> Shall we enable the cartridge agent debug logs. Then we can compare >>>>>> both CEP and agent logs with the time stamp. Then we can narrow down >>>>>> where >>>>>> the actual problem is. >>>>>> >>>>> >>>>> Manula, cep-trace log have the values sent by cartridge agent also >>>>> right? >>>>> >>>>> >>>>>> >>>>>> Thanks ! >>>>>> >>>>>> On Mon, Oct 27, 2014 at 5:06 PM, Rajkumar Rajaratnam < >>>>>> rajkum...@wso2.com> wrote: >>>>>> >>>>>>> >>>>>>> >>>>>>> On Mon, Oct 27, 2014 at 12:53 PM, Manula Chathurika Thantriwatte < >>>>>>> manu...@wso2.com> wrote: >>>>>>> >>>>>>>> Hi Raj, >>>>>>>> >>>>>>>> Is this happen frequently or intermittently ? Once I go though the >>>>>>>> logs I'm unable to find the high input values for load average. But >>>>>>>> output >>>>>>>> have high values. >>>>>>>> >>>>>>>> 22:10:38,698 [-] [Siddhi-Scheduler-pool-10-thread-1] INFO >>>>>>>> EVENT_TRACE_LOGGER TenantId=-1234 : Output Event Adaptor : >>>>>>>> JMSOutputAdaptor, sent >>>>>>>> >>>>>>>> {"org.apache.stratos.messaging.event.health.stat.SecondDerivativeOfLoadAverageEvent":{"message":{"clusterId":"php.php.domain","networkPartitionId":"","value":"2000000.0"}}} >>>>>>>> >>>>>>> >>>>>>> This is happening intermittently. Experienced now too. This is a >>>>>>> serious issue. We need to somehow sort this out. >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> Also networkPartitionId is empty. In the current execution plans it >>>>>>>> took networkPartitionId for the calculation. IMO we need to update the >>>>>>>> CEP >>>>>>>> execution plans for 4.1.0. >>>>>>>> >>>>>>>> Thanks ! >>>>>>>> >>>>>>>> >>>>>>>> On Sun, Oct 26, 2014 at 8:49 AM, Manula Chathurika Thantriwatte < >>>>>>>> manu...@wso2.com> wrote: >>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> Thanks Raj. I'll go through them. >>>>>>>>> >>>>>>>>> Thanks ! >>>>>>>>> >>>>>>>>> On Sun, Oct 26, 2014 at 8:33 AM, Rajkumar Rajaratnam < >>>>>>>>> rajkum...@wso2.com> wrote: >>>>>>>>> >>>>>>>>>> Hi Manula, >>>>>>>>>> >>>>>>>>>> You can see those values in the same log I attached in the >>>>>>>>>> previous mail. >>>>>>>>>> >>>>>>>>>> Thanks. >>>>>>>>>> >>>>>>>>>> On Sun, Oct 26, 2014 at 8:29 AM, Rajkumar Rajaratnam < >>>>>>>>>> rajkum...@wso2.com> wrote: >>>>>>>>>> >>>>>>>>>>> Hi Chamila, >>>>>>>>>>> >>>>>>>>>>> I couldn't find any large values sent by agent. There are traces >>>>>>>>>>> for CEP sending large values. Find the cep trace log here [1] >>>>>>>>>>> >>>>>>>>>>> 1. >>>>>>>>>>> https://drive.google.com/file/d/0B1haIleqJMHIS3FFVFpfa0JlbVE/view?usp=sharing >>>>>>>>>>> >>>>>>>>>>> Thanks. >>>>>>>>>>> >>>>>>>>>>> On Sun, Oct 26, 2014 at 8:23 AM, Manula Chathurika Thantriwatte >>>>>>>>>>> <manu...@wso2.com> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> Shall we enable the cartridge agent debug logs and see what are >>>>>>>>>>>> the load average values sent from the agent. If those values are >>>>>>>>>>>> normal >>>>>>>>>>>> then we can narrow down to the CEP. >>>>>>>>>>>> >>>>>>>>>>>> Thanks ! >>>>>>>>>>>> >>>>>>>>>>>> On Sat, Oct 25, 2014 at 10:43 PM, Chamila De Alwis < >>>>>>>>>>>> chami...@wso2.com> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi Raj, >>>>>>>>>>>>> >>>>>>>>>>>>> Can you inspect the CEP trace logs for the same time period? >>>>>>>>>>>>> It will have the events published from the agent. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Regards, >>>>>>>>>>>>> Chamila de Alwis >>>>>>>>>>>>> Software Engineer | WSO2 | +94772207163 >>>>>>>>>>>>> Blog: code.chamiladealwis.com >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On Sat, Oct 25, 2014 at 10:39 PM, Rajkumar Rajaratnam < >>>>>>>>>>>>> rajkum...@wso2.com> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>> >>>>>>>>>>>>>> I am doing a test round with M3 with python agent. And faced >>>>>>>>>>>>>> this very strange scenario, $Subject. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,693] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - Load >>>>>>>>>>>>>> average >>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,693] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>> - Grad of >>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] -2000.0 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,694] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - Load >>>>>>>>>>>>>> average >>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,694] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>> - Grad of >>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 0.0 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,694] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - Load >>>>>>>>>>>>>> average >>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,694] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>> - Avg >>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 1.9090909 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,694] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - Load >>>>>>>>>>>>>> average >>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,694] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>> - Avg >>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value] >>>>>>>>>>>>>> 28.718182 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,695] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>> - Second >>>>>>>>>>>>>> Derivation of Memory Consumption event: [cluster] php.php.domain >>>>>>>>>>>>>> [value] >>>>>>>>>>>>>> 200000.0 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,695] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>> - Grad of >>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 0.0 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,695] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - Load >>>>>>>>>>>>>> average >>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,695] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>> - Second >>>>>>>>>>>>>> Derivation of load avg event: [cluster] php.php.domain [value] >>>>>>>>>>>>>> 0.0 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,695] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - Load >>>>>>>>>>>>>> average >>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,696] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>> - Second >>>>>>>>>>>>>> Derivation of load avg event: [cluster] php.php.domain [value] >>>>>>>>>>>>>> 0.0 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,696] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - Load >>>>>>>>>>>>>> average >>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,696] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>> - Avg >>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 3.3636363 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,696] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - Load >>>>>>>>>>>>>> average >>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,696] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>> - Grad of >>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value] >>>>>>>>>>>>>> 0.11670045 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,696] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - >>>>>>>>>>>>>> Memory >>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub >>>>>>>>>>>>>> cluster] KubGrp1 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>> - Avg >>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value] 28.7 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - >>>>>>>>>>>>>> Memory >>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub >>>>>>>>>>>>>> cluster] KubGrp1 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>> - Second >>>>>>>>>>>>>> Derivation of Memory Consumption event: [cluster] php.php.domain >>>>>>>>>>>>>> [value] 0.0 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - >>>>>>>>>>>>>> Memory >>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub >>>>>>>>>>>>>> cluster] KubGrp1 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>> - Avg >>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value] 28.7 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - >>>>>>>>>>>>>> Memory >>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub >>>>>>>>>>>>>> cluster] KubGrp1 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>> - Avg >>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 5.0 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - Load >>>>>>>>>>>>>> average >>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,697] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>> - Grad of >>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 2000.0 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,698] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - Load >>>>>>>>>>>>>> average >>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,698] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>> - Avg >>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value] 28.7 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,698] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - >>>>>>>>>>>>>> Memory >>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub >>>>>>>>>>>>>> cluster] KubGrp1 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,698] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>> - Grad of >>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value] 0.0 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,698] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - >>>>>>>>>>>>>> Memory >>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub >>>>>>>>>>>>>> cluster] KubGrp1 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,699] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>> - Second >>>>>>>>>>>>>> Derivation of Memory Consumption event: [cluster] php.php.domain >>>>>>>>>>>>>> [value] 0.0 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,699] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - >>>>>>>>>>>>>> Memory >>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub >>>>>>>>>>>>>> cluster] KubGrp1 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,700] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>> - Second >>>>>>>>>>>>>> Derivation of load avg event: [cluster] php.php.domain [value] >>>>>>>>>>>>>> 2000000.0 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,700] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - Load >>>>>>>>>>>>>> average >>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,700] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>> - Grad of >>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 13500.0 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,700] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - Load >>>>>>>>>>>>>> average >>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,701] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>> - Grad of >>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value] 50.0 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,701] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - >>>>>>>>>>>>>> Memory >>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub >>>>>>>>>>>>>> cluster] KubGrp1 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,701] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>> - Avg >>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value] >>>>>>>>>>>>>> 28.738462 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,701] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - >>>>>>>>>>>>>> Memory >>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub >>>>>>>>>>>>>> cluster] KubGrp1 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,702] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>> - Avg >>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 2.3333333 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,702] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - Load >>>>>>>>>>>>>> average >>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,702] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>> - Grad of >>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 0.0 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,702] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - Load >>>>>>>>>>>>>> average >>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,703] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>> - Grad of >>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value] 0.0 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,703] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - >>>>>>>>>>>>>> Memory >>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub >>>>>>>>>>>>>> cluster] KubGrp1 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,703] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>> - Avg >>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 31.0 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,703] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - Load >>>>>>>>>>>>>> average >>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,704] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>> - Grad of >>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value] 0.0 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,704] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - >>>>>>>>>>>>>> Memory >>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub >>>>>>>>>>>>>> cluster] KubGrp1 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,704] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>> - Avg >>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 39.0 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,704] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - Load >>>>>>>>>>>>>> average >>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,704] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>> - Avg >>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] 39.0 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,705] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - Load >>>>>>>>>>>>>> average >>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,706] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>> - Grad of >>>>>>>>>>>>>> load avg event: [cluster] php.php.domain [value] -11000.0 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,706] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - Load >>>>>>>>>>>>>> average >>>>>>>>>>>>>> stats are reset, ready to do scale check [kub cluster] KubGrp1 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,708] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>> - Grad of >>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value] 0.0 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,708] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - >>>>>>>>>>>>>> Memory >>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub >>>>>>>>>>>>>> cluster] KubGrp1 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,709] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesClusterMonitor} >>>>>>>>>>>>>> - Avg >>>>>>>>>>>>>> Memory Consumption event: [cluster] php.php.domain [value] 28.8 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,709] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.KubernetesClusterContext} - >>>>>>>>>>>>>> Memory >>>>>>>>>>>>>> consumption stats are reset, ready to do scale check [kub >>>>>>>>>>>>>> cluster] KubGrp1 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,811] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.cloud.controller.util.PodActivationWatcher} - >>>>>>>>>>>>>> PodActivationWatcher running : Running >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,942] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.cloud.controller.util.PodActivationWatcher} - >>>>>>>>>>>>>> PodActivationWatcher running : Running >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:38,945] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.cloud.controller.util.PodActivationWatcher} - >>>>>>>>>>>>>> PodActivationWatcher running : Running >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,712] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesServiceClusterMonitor} >>>>>>>>>>>>>> - >>>>>>>>>>>>>> KubernetesServiceClusterMonitor is running.. >>>>>>>>>>>>>> KubernetesServiceClusterMonitor [ >>>>>>>>>>>>>> kubernetesHostClusterId=KubGrp1, >>>>>>>>>>>>>> clusterId=php.php.domain, serviceId=php] >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,713] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesServiceClusterMonitor} >>>>>>>>>>>>>> - >>>>>>>>>>>>>> Running min check for [kub-cluster] : KubGrp1 [cluster] : >>>>>>>>>>>>>> php.php.domain >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,713] INFO >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - Running minimum >>>>>>>>>>>>>> rule: >>>>>>>>>>>>>> [kub-cluster] KubGrp1 [cluster] php.php.domain >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,714] INFO >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [min-check] >>>>>>>>>>>>>> [cluster] : >>>>>>>>>>>>>> php.php.domain [Replicas] nonTerminated : 3 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,714] INFO >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [min-check] >>>>>>>>>>>>>> [cluster] : >>>>>>>>>>>>>> php.php.domain [Replicas] minReplicas : 3 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,715] INFO >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - Running obsolete >>>>>>>>>>>>>> containers >>>>>>>>>>>>>> rule [kub-cluster] : KubGrp1 [cluster] : php.php.domain >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,715] INFO >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [obsolete-check] >>>>>>>>>>>>>> [cluster] >>>>>>>>>>>>>> : php.php.domain [Replicas] obsoleteReplicas : 0 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,715] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.AutoscalerRuleEvaluator} - >>>>>>>>>>>>>> Minimum >>>>>>>>>>>>>> check executed for : >>>>>>>>>>>>>> org.apache.stratos.autoscaler.KubernetesClusterContext@21d906d0 >>>>>>>>>>>>>> >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,715] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesServiceClusterMonitor} >>>>>>>>>>>>>> - >>>>>>>>>>>>>> flag of rifReset : true flag of memoryConsumptionReset : true >>>>>>>>>>>>>> flag of >>>>>>>>>>>>>> loadAverageReset : true >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,715] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.monitor.KubernetesServiceClusterMonitor} >>>>>>>>>>>>>> - >>>>>>>>>>>>>> Running scale check for [kub-cluster] : KubGrp1 [cluster] : >>>>>>>>>>>>>> php.php.domain >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,716] INFO >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - Running scaling >>>>>>>>>>>>>> rule >>>>>>>>>>>>>> [kub-cluster] : KubGrp1 [cluster] : php.php.domain >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,716] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleTasksDelegator} - >>>>>>>>>>>>>> Predicting the >>>>>>>>>>>>>> value, [average]: 0.0 , [gradient]: 0.0 , [second derivative]: >>>>>>>>>>>>>> 0.0 , [time >>>>>>>>>>>>>> intervals]: 1 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,716] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleTasksDelegator} - >>>>>>>>>>>>>> Predicting the >>>>>>>>>>>>>> value, [average]: 28.8 , [gradient]: 0.0 , [second derivative]: >>>>>>>>>>>>>> 0.0 , [time >>>>>>>>>>>>>> intervals]: 1 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,717] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleTasksDelegator} - >>>>>>>>>>>>>> Predicting the >>>>>>>>>>>>>> value, [average]: 39.0 , [gradient]: -11000.0 , [second >>>>>>>>>>>>>> derivative]: >>>>>>>>>>>>>> 2000000.0 , [time intervals]: 1 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,717] INFO >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling] >>>>>>>>>>>>>> [cluster] : >>>>>>>>>>>>>> php.php.domain [Replicas] minReplicas : 3 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,717] INFO >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling] >>>>>>>>>>>>>> [cluster] : >>>>>>>>>>>>>> php.php.domain [Replicas] maxReplicas : 10 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718] INFO >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling] >>>>>>>>>>>>>> [cluster] : >>>>>>>>>>>>>> php.php.domain [Replicas] nonTerminated : 3 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718] INFO >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling] >>>>>>>>>>>>>> [cluster] : >>>>>>>>>>>>>> php.php.domain [Replicas] activeReplicas : 3 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718] INFO >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling] >>>>>>>>>>>>>> [cluster] : >>>>>>>>>>>>>> php.php.domain [RequestInFlight] predicted value : 0.0 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718] INFO >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling] >>>>>>>>>>>>>> [cluster] : >>>>>>>>>>>>>> php.php.domain [RequestInFlight] upper limit : 80.0 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718] INFO >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling] >>>>>>>>>>>>>> [cluster] : >>>>>>>>>>>>>> php.php.domain [RequestInFlight] lower limit : 5.0 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718] INFO >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling] >>>>>>>>>>>>>> [cluster] : >>>>>>>>>>>>>> php.php.domain [MemoryConsumption] predicted value : >>>>>>>>>>>>>> 28.799999237060547 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718] INFO >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling] >>>>>>>>>>>>>> [cluster] : >>>>>>>>>>>>>> php.php.domain [MemoryConsumption] upper limit : 80.0 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718] INFO >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling] >>>>>>>>>>>>>> [cluster] : >>>>>>>>>>>>>> php.php.domain [MemoryConsumption] lower limit : 15.0 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,718] INFO >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling] >>>>>>>>>>>>>> [cluster] : >>>>>>>>>>>>>> php.php.domain [LoadAverage] predicted value : 989039.0 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,719] INFO >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling] >>>>>>>>>>>>>> [cluster] : >>>>>>>>>>>>>> php.php.domain [LoadAverage] upper limit : 180.0 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,719] INFO >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling] >>>>>>>>>>>>>> [cluster] : >>>>>>>>>>>>>> php.php.domain [LoadAverage] lower limit : 20.0 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,719] INFO >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling] >>>>>>>>>>>>>> [cluster] : >>>>>>>>>>>>>> php.php.domain scale-up action : true >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,719] INFO >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling] >>>>>>>>>>>>>> [cluster] : >>>>>>>>>>>>>> php.php.domain scale-down action : false >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,719] INFO >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling] >>>>>>>>>>>>>> [cluster] : >>>>>>>>>>>>>> php.php.domain [LoadAverage] predicted replicas : 16484 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,720] INFO >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling] >>>>>>>>>>>>>> [cluster] : >>>>>>>>>>>>>> php.php.domain predicted replicas > max replicas : >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,720] INFO >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling] >>>>>>>>>>>>>> Decided to >>>>>>>>>>>>>> scale-up : [cluster] : php.php.domain >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,720] INFO >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.rule.RuleLog} - [scaling-up] >>>>>>>>>>>>>> [cluster] : >>>>>>>>>>>>>> php.php.domain valid number of replicas to expand : 10 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,720] INFO >>>>>>>>>>>>>> {org.apache.stratos.autoscaler.client.cloud.controller.CloudControllerClient} >>>>>>>>>>>>>> - Updating kubernetes replication controller via cloud >>>>>>>>>>>>>> controller: >>>>>>>>>>>>>> [cluster] php.php.domain [replicas] 10 >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,734] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.cloud.controller.impl.CloudControllerServiceImpl} >>>>>>>>>>>>>> - >>>>>>>>>>>>>> CloudControllerServiceImpl:updateContainers for cluster : >>>>>>>>>>>>>> php.php.domain >>>>>>>>>>>>>> TID: [0] [STRATOS] [2014-10-25 22:10:43,758] DEBUG >>>>>>>>>>>>>> {org.apache.stratos.cloud.controller.impl.CloudControllerServiceImpl} >>>>>>>>>>>>>> - >>>>>>>>>>>>>> Cloud Controller is delegating request to update a replication >>>>>>>>>>>>>> controller >>>>>>>>>>>>>> php.php.domain to Kubernetes layer. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> I couldn't investigate the agent side, since containers are >>>>>>>>>>>>>> destroyed. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Any idea? >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks. >>>>>>>>>>>>>> >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> Rajkumar Rajaratnam >>>>>>>>>>>>>> Software Engineer | WSO2, Inc. >>>>>>>>>>>>>> Mobile +94777568639 | +94783498120 >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> Regards, >>>>>>>>>>>> Manula Chathurika Thantriwatte >>>>>>>>>>>> Software Engineer >>>>>>>>>>>> WSO2 Inc. : http://wso2.com >>>>>>>>>>>> lean . enterprise . middleware >>>>>>>>>>>> >>>>>>>>>>>> email : manu...@wso2.com / man...@apache.org >>>>>>>>>>>> phone : +94 772492511 >>>>>>>>>>>> blog : http://manulachathurika.blogspot.com/ >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Rajkumar Rajaratnam >>>>>>>>>>> Software Engineer | WSO2, Inc. >>>>>>>>>>> Mobile +94777568639 | +94783498120 >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Rajkumar Rajaratnam >>>>>>>>>> Software Engineer | WSO2, Inc. >>>>>>>>>> Mobile +94777568639 | +94783498120 >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Regards, >>>>>>>>> Manula Chathurika Thantriwatte >>>>>>>>> Software Engineer >>>>>>>>> WSO2 Inc. : http://wso2.com >>>>>>>>> lean . enterprise . middleware >>>>>>>>> >>>>>>>>> email : manu...@wso2.com / man...@apache.org >>>>>>>>> phone : +94 772492511 >>>>>>>>> blog : http://manulachathurika.blogspot.com/ >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Regards, >>>>>>>> Manula Chathurika Thantriwatte >>>>>>>> Software Engineer >>>>>>>> WSO2 Inc. : http://wso2.com >>>>>>>> lean . enterprise . middleware >>>>>>>> >>>>>>>> email : manu...@wso2.com / man...@apache.org >>>>>>>> phone : +94 772492511 >>>>>>>> blog : http://manulachathurika.blogspot.com/ >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Rajkumar Rajaratnam >>>>>>> Software Engineer | WSO2, Inc. >>>>>>> Mobile +94777568639 | +94783498120 >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Regards, >>>>>> Manula Chathurika Thantriwatte >>>>>> Software Engineer >>>>>> WSO2 Inc. : http://wso2.com >>>>>> lean . enterprise . middleware >>>>>> >>>>>> email : manu...@wso2.com / man...@apache.org >>>>>> phone : +94 772492511 >>>>>> blog : http://manulachathurika.blogspot.com/ >>>>>> >>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Rajkumar Rajaratnam >>>>> Software Engineer | WSO2, Inc. >>>>> Mobile +94777568639 | +94783498120 >>>>> >>>> >>>> >>>> >>>> -- >>>> Regards, >>>> Manula Chathurika Thantriwatte >>>> Software Engineer >>>> WSO2 Inc. : http://wso2.com >>>> lean . enterprise . middleware >>>> >>>> email : manu...@wso2.com / man...@apache.org >>>> phone : +94 772492511 >>>> blog : http://manulachathurika.blogspot.com/ >>>> >>>> >>>> >>>> >>>> >>>> >>>> -- >>>> Best Regards, >>>> Nirmal >>>> >>>> Nirmal Fernando. >>>> PPMC Member & Committer of Apache Stratos, >>>> Senior Software Engineer, WSO2 Inc. >>>> >>>> Blog: http://nirmalfdo.blogspot.com/ >>>> >>> >>> >>> >>> -- >>> Imesh Gunaratne >>> >>> Technical Lead, WSO2 >>> Committer & PMC Member, Apache Stratos >>> >> >> >> >> -- >> Raj >> > > > > -- > > Gayan Gunarathne > Technical Lead > WSO2 Inc. (http://wso2.com) > email : gay...@wso2.com | mobile : +94 766819985 > > -- Best Regards, Nirmal Nirmal Fernando. PPMC Member & Committer of Apache Stratos, Senior Software Engineer, WSO2 Inc. Blog: http://nirmalfdo.blogspot.com/