Great to hear that. Thanks.
On Sat, Mar 22, 2014 at 1:53 AM, Asiri Liyana Arachchi <[email protected]>wrote: > I've submit the proposal for "Improvements to Autoscaling for Apache > Stratos" project at google-melange. > > Here is the link > > > https://www.google-melange.com/gsoc/proposal/review/student/google/gsoc2014/asiria/5629499534213120 > > > Regards > Asiri > > > > On Tue, Mar 18, 2014 at 4:29 AM, Asiri Liyana Arachchi < > [email protected]> wrote: > >> Thanks a lot for the elaborated reply. >> >> It helped a lot in getting familiar with Drools by running samples as >> you've pointed. And I've built the code base. >> >> After going through scaling.drl >> (products/autoscaler/modules/distribution/src/main/conf/scaling.drl) it was >> clear that currently stratos uses >> RuleTasksDelegator.getPredictedValueForNextMinute() method to compare, stat >> values against the thresholds. >> >> *Approach on deciding the number of instances that might need to handle >> the load:* >> >> Using existing method on predicting next minute Requests inflight, Load >> average and Memory Consumption. >> >> - Assumption: current thresholds of those metrics are the optimal >> values for an instance. >> - Based on that implementing a simple algorithm to decide, how many >> number of instances that might need for the next minute using predicted >> values for those metrics. >> - That algorithm will be implemented in such a way that it always >> will keep the instances under thresholds (or near thresholds ) of one or >> more metrics , with out exceeding them. >> - Assumption : metrics act inverse or direct proportionally when >> instances are spawned. (for an ex. load is equally distributed among all >> the instances + newly spawned instances. ) >> >> *Predict the load according to a schedule defined by end user * >> >> *Does this mean providing a functionality in web UI to define a schedule >> and make it active? *It's not clear to me. >> *Can this be achieved by generating an auto scale policy xml with user >> defined thresholds similar to how it's done currently and making it >> possible to override the *auto-scaling* algorithm in use when it's >> needed (like in a specific time *which is already defined) ? . >> >> Thanks >> Asiri >> >> >> >> >> >> >> >> >> >> >> >> On Wed, Mar 12, 2014 at 8:05 AM, Lahiru Sandaruwan <[email protected]>wrote: >> >>> Hi Asiri, >>> >>> It is a pleasure to see your interest. Sorry for the late reply. I >>> missed the mail. >>> >>> Get the code base and build as a starting point for Stratos. >>> >>> You will not find Drools hard, after running some samples. [1] looks >>> like a good sample. You can just run those in WSO2 BRS. You can use your >>> Java knowledge as we can write Java code in "then" section. >>> >>> AMQP knowledge means you have to understand pub/sub model with topics. >>> Conceptually thats it. In addition, handling subs/pubs using java codes. >>> >>> Great research, find the comments inline. >>> >>> >>> On Tue, Mar 11, 2014 at 11:23 AM, Asiri Liyana Arachchi < >>> [email protected]> wrote: >>> >>>> 1. Improve Auto-scaling to predict the number of instances required in >>>> the next time interval. >>>> >>>> As far as I understood, this project aims at introducing a new auto >>>> scaling strategy apart from the threshold based auto scaling which is >>>> currently in use, to stratos making it more proactive on auto-scaling. >>>> >>> >>> Correct. So system should scale, understanding the load and hence the >>> number of instances that would require to handle that load. >>> >>> We have 3 types of information about load, and should consider all 3 for >>> our decision. >>> >>> - Requests inflight(Information about how many requests are waiting >>> to get the response) >>> - Load average of cartridge instances running >>> - Memory consumption of cartridge instances running >>> >>> >>> To do that there are several strategies suggested. >>>> >>>> 1. Kalman Filter >>>> 2. Control theory >>>> 3. Time Series Analysis. >>>> 4. FFT >>>> >>>> As I've gone through these techniques as for now I felt that Kalman >>>> Filter would be the most viable candidate and it can be used to address >>>> this issue effectively.There is an apache API for Kalman filter [1]. >>>> >>> >>> We should find an efficient, yet simplest way to get the job done. We >>> currently use S = u*t + 0.5 *a*t*t prediction(motion) equation. This is one >>> of the equations that Kalman filter used to do prediction. But with this, >>> we have to compare with a threshold to take the decision. >>> >>> We receive second derivative, gradient and average values at a given >>> time. Lets say we time interval we consider is minute. So we can predict >>> the load in the next minute using them. >>> Also we know the number of instances that are running at the moment. The >>> algorithm does not need to be complex. It should be just intelligent enough >>> to find the matching number of instances that should be there in the next >>> minute. >>> >>> [1] https://docs.wso2.org/display/BRS200/Sample+Rule+Definition >>> >>> Thanks. >>> >>>> >>>> But I think selecting an auto scaling algorithm would involve more of >>>> research and testing. Even selecting metrics to predict on will also be >>>> challenging because some of the metrics for an example *load average >>>> *depends >>>> on autos-scalling causing predictions to deviate from the actual values. >>>> >>> I would appreciate if you can comment on this. >>>> >>>> [1] : >>>> http://commons.apache.org/proper/commons-math/apidocs/org/apache/commons/math3/filter/KalmanFilter.html >>>> >>>> >>>> >>>> Thanks >>>> Asiri >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> On Thu, Mar 6, 2014 at 7:38 AM, Udara Liyanage <[email protected]> wrote: >>>> >>>>> Hi Asiri, >>>>> >>>>> Glad to hear your interest on Stratos. I don't think it will take more >>>>> than few days to learn drools and amqp. You will be able to do it within >>>>> given time period. >>>>> Happy to see your project proposal soon. >>>>> >>>>> Touched, not typed. Erroneous words are a feature, not a typo. >>>>> On Mar 6, 2014 7:13 AM, "Asiri Liyana Arachchi" <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi all, >>>>>> >>>>>> I'm Asiri Liyana Arachchi , third year student studying Computer >>>>>> Science and Engineering in University of Moratuwa , Sri Lanka. >>>>>> I would like to start contributing towards the project $subject .I've >>>>>> gone through the resources about this project including stratos >>>>>> documentation and the code-base. >>>>>> >>>>>> As expected I'm familiur with java , json and SOA . I would like to >>>>>> know how well and in what cases Drools and APQM skills are required. Also >>>>>> would it be feasible to complete the project in the projects limited >>>>>> time, >>>>>> considered that the Drools and APQM are to be learnt along with the total >>>>>> work of the project. >>>>>> >>>>>> >>>>>> Thanks. >>>>>> >>>>>> Asiri >>>>>> >>>>>> >>>> >>> >>> >>> -- >>> -- >>> Lahiru Sandaruwan >>> Software Engineer, >>> Platform Technologies, >>> WSO2 Inc., http://wso2.com >>> lean.enterprise.middleware >>> >>> email: [email protected] cell: (+94) 773 325 954 >>> blog: http://lahiruwrites.blogspot.com/ >>> twitter: http://twitter.com/lahirus >>> linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146 >>> >>> >> > -- -- Lahiru Sandaruwan Software Engineer, Platform Technologies, WSO2 Inc., http://wso2.com lean.enterprise.middleware email: [email protected] cell: (+94) 773 325 954 blog: http://lahiruwrites.blogspot.com/ twitter: http://twitter.com/lahirus linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
