Would it also make sense to have a /dependencies project where we can merge the libs coming from carbon kernel, there we can group stuff user-mgt, registry, and kernel (carbon.core, carbon.utils etc). One advantage is it would be much easier for developers to have one or two coarse-grained libs instead of the many jars in the classpath, just a thought .. !
On Sat, Jul 6, 2013 at 9:26 AM, Lakmal Warusawithana <[email protected]>wrote: > Hi Sameera, > > Yes have to create more coarse-grained components. Will do after the > initial refactor completed. > > thanks > > > On Fri, Jul 5, 2013 at 9:37 PM, Sameera Jayasoma < > [email protected]> wrote: > >> Hi Lakmal >> >> IMHO we need to consider the option of having more coarse-grained >> components. If you look at the existing components and feature, there are >> too fine-grained. This is true even in the previous Stratos and Carbon code >> bases. Lets not make the same mistake again here. If we go ahead with >> fine-grained components then it will become a maintenance night-mare in the >> future. >> >> So I would suggest few logical components and few logical features(i.e >> Installable Units). >> >> Thanks, >> Sameera. >> >> >> On Tue, Jul 2, 2013 at 9:24 AM, Lakmal Warusawithana <[email protected]>wrote: >> >>> Hi all, >>> >>> I will start separate threads to discuss each item in detail. What I can >>> see milestone one should be, changing code to apache and re-factor the code >>> to new structure. After discuss in detail will prioritize and then finalize >>> the next milestone items. >>> >>> Current Stratos we use osgi model and will continue it. We will simplify >>> folder structure as below. >>> >>> >>> / >>> -Components >>> -- all osgi components >>> - Features (logically bundle components together) >>> -- adc >>> -- load balancer >>> -- auto scaller >>> -- manager >>> -- cloud controller >>> -- cli >>> -- cartridge agent >>> - Products (bundle several features and make a product) >>> - Service-stubs >>> -- all service stubs >>> - Tools >>> -- tools for create cartridges >>> >>> >>> Please share your thoughts. >>> >>> >>> thanks >>> >>> >>> On Thu, Jun 27, 2013 at 11:50 AM, Dharshana kasun Warusavitharana < >>> [email protected]> wrote: >>> >>>> +1 for idea of load balance within static members. It allows to have >>>> more openings and be more flexible. >>>> >>>> >>>> Thank You, >>>> Dharshana. >>>> >>>> >>>> On Wed, Jun 26, 2013 at 7:39 PM, sanjeewa rangana < >>>> [email protected]> wrote: >>>> >>>>> >>>>> >>>>> >>>>> On Wed, Jun 26, 2013 at 10:35 AM, Nirmal Fernando < >>>>> [email protected]> wrote: >>>>> >>>>>> Hi Sanjeewa, >>>>>> >>>>>> >>>>>> On Tue, Jun 25, 2013 at 12:15 PM, sanjeewa rangana < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Mon, Jun 24, 2013 at 6:16 PM, Lakmal Warusawithana < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Hi All, >>>>>>>> >>>>>>>> I hope all commiters / folks are in the mailing list now. Here I >>>>>>>> have listed below, some identified improvements, features that like to >>>>>>>> have >>>>>>>> in Stratos. >>>>>>>> >>>>>>>> Any other improvements/features are mostly welcome and will add >>>>>>>> those to the bellow list, then we can discuss and then will create >>>>>>>> milestones plan for releases. ( We will commit current code based soon >>>>>>>> after we got the git repository available) >>>>>>>> >>>>>>>> Before that if some one want to get quick overview of current >>>>>>>> release/architecture of Stratos please see [1] for my reason blog post >>>>>>>> or >>>>>>>> more detail [2] with current product documentation, which will migrate >>>>>>>> to >>>>>>>> Apache soon. Also we will organize some hangout sessions to explain >>>>>>>> current >>>>>>>> architecture and code based, and it may helpful other commierts to come >>>>>>>> up-to speed quickly. >>>>>>>> >>>>>>>> Here the list; >>>>>>>> >>>>>>>> - Re-factor current code to Apache >>>>>>>> >>>>>>>> First we will move current code based to apache and then start >>>>>>>> re-factoring to apache and make a release without adding any other >>>>>>>> improvement or feature. >>>>>>>> >>>>>>>> >>>>>>>> - IaaS Independent cartridge for Linux. >>>>>>>> >>>>>>>> With the current architecture when you create a cartridge its >>>>>>>> depend on the IaaS. For example If you create a cartridge in EC2, we >>>>>>>> have >>>>>>>> to make an AMI cartridge. Likewise you have to create cartridges >>>>>>>> (say PHP >>>>>>>> cartridge) for each and every IaaSs for support the IaaSs. As a >>>>>>>> solution we >>>>>>>> can introduce LXC based cartridges (which IaaS independent cartridges) >>>>>>>> and >>>>>>>> can be consider LXC is the default Stratos runtime container for Linux >>>>>>>> cartridges. Also this will provide more scallability to the PaaS (I >>>>>>>> will >>>>>>>> start separate mail thread with more detail on this) >>>>>>>> >>>>>>>> >>>>>>>> - Support Windows based cartridges. >>>>>>>> >>>>>>>> If IaaS support windows VM then we can have windows based >>>>>>>> cartridges. It will lead to have .net cartridges >>>>>>>> >>>>>>>> >>>>>>>> - Expand auto-scaling algorithm >>>>>>>> >>>>>>>> Currently auto scaling decision take based on looking at length of >>>>>>>> the http/https request queue. We have to expand this to consider CPU >>>>>>>> usage, >>>>>>>> memory utilization ..etc of the cartridges. >>>>>>>> >>>>>>>> Before we introduce que length based decision making mechanism we >>>>>>> had load average based decision making mechanism. Each server need to >>>>>>> expose some API(some service) to get load average of given server. Load >>>>>>> calculation can be different from one server to other. Load balancer >>>>>>> will >>>>>>> do web service call to those end points and update load average values >>>>>>> of >>>>>>> member list periodically. >>>>>>> >>>>>> >>>>>> We will separate out the load balancer logic and the auto-scaling >>>>>> logic soon. This would require some code level changes. But the vision is >>>>>> there. >>>>>> >>>>>> In brief, we should build APIs in Auto-scaler, so that the external >>>>>> components could provide/insert information that matters the decision of >>>>>> scaling up/down, depending on the algorithm used. >>>>>> >>>>>> Normally that service returns integer value(let say 0 to 10) and we >>>>>>> can get some idea about load from that. When it comes to traffic route >>>>>>> we >>>>>>> will give high priority to server with minimum load and so on. So that >>>>>>> type >>>>>>> of implementation would be ideal for this case as well. Synapse member >>>>>>> selection algorithm can be configurable and we can add new algorithm for >>>>>>> this if need. >>>>>>> >>>>>> >>>>>> This is an extension point, one could leverage. >>>>>> >>>>>>> >>>>>>>> - TCP level load balance. >>>>>>>> >>>>>>>> Current we only doing http/https based load balancing but need to >>>>>>>> expand this to do TCP level load balancing. >>>>>>>> >>>>>>>> This would be again nice feature IMO. Also i think we need >>>>>>> considerable amount of synapse transport level improvements to implement >>>>>>> this. WDYT? >>>>>>> >>>>>> >>>>>> We may require to write a TCP load balance endpoint, if one is not >>>>>> already available (I couldn't find any reference). >>>>>> >>>>>>> >>>>>>>> - Health monitoring >>>>>>>> >>>>>>>> Current release we haven't much in the cartridge heath monitoring >>>>>>>> but its very important and need to have in the future release. >>>>>>>> >>>>>>>> If we implemented above mentioned load average calculation >>>>>>> mechanism then we can do this easily. >>>>>>> >>>>>> >>>>>> Yes, health monitor would ideally consider the load average of the >>>>>> Cartridge instances, memory consumption etc. and transit the state >>>>>> according to a pre-defined curriculum, IMO. >>>>>> >>>>>> We can call that end point and get whatever we need. Also it would >>>>>>> be ideal if we can have some simple user interface for load balancer to >>>>>>> view active services and subscribed nodes and their status. Or else we >>>>>>> can >>>>>>> use cluster message and set status of server as member attribute. >>>>>>> >>>>>> >>>>>>> Also i would like to suggest static load balance endpoint support >>>>>>> for load balancer(as a suggestion). Let say some user have fixed set of >>>>>>> back end servers and need to route traffic to them through load >>>>>>> balancer. >>>>>>> That would be useful feature and we can consider it when we design >>>>>>> milestone plan. >>>>>>> >>>>>> >>>>>> I'm afraid this won't be a use-case when you consider Apache Stratos. >>>>>> Apache Stratos is a fully dynamic environment i.e. all the clusters get >>>>>> build, as and when required. We don't have a concept call static >>>>>> endpoints >>>>>> in the context of Stratos. >>>>>> >>>>> Yes i can understand. Thought it would be more useful if someone try >>>>> to load balance within static members. >>>>> >>>>>> >>>>>> Sample configuration would be as follows(It tells everything about >>>>>>> implementation). Sometimes back me and azeez had offline discussion >>>>>>> about >>>>>>> this. >>>>>>> >>>>>>> TestService { >>>>>>> hosts server.test.com; >>>>>>> servers { >>>>>>> server1 { >>>>>>> ip = 192.0.0.1; >>>>>>> http = 80; >>>>>>> https = 443; >>>>>>> } >>>>>>> >>>>>>> server2 { >>>>>>> ip = 192.0.0.2; >>>>>>> http = 80; >>>>>>> https = 443; >>>>>>> } >>>>>>> } >>>>>>> } >>>>>>> >>>>>>>> >>>>>>>> - Improving ADC (Artifact Distribution Coordinator) >>>>>>>> >>>>>>>> Current release we only support git based artifact distribution and >>>>>>>> need to provide more scalable deployment options >>>>>>>> >>>>>>>> >>>>>>>> - Light weight cartridge agent >>>>>>>> >>>>>>>> Current load balancer used tribes based clustering. To support >>>>>>>> tribes clustering we have to use java based cartridge agent and it >>>>>>>> will put >>>>>>>> some unnecessary heavy load to non java cartridges like PHP. If we can >>>>>>>> moved to Hazelcast Clustering we can have python based light weight >>>>>>>> cartridge agent across all type of cartridges. >>>>>>>> >>>>>>>> >>>>>>>> - Adding more and more cartridges >>>>>>>> >>>>>>>> We can have more and more cartridges like Python, Ruby, Node.js, >>>>>>>> Mongo etc. >>>>>>>> >>>>>>>> >>>>>>>> thanks >>>>>>>> >>>>>>>> [1] >>>>>>>> http://lakmalsview.blogspot.com/2013/06/wso2-stratos-200-is-released.html >>>>>>>> [2] >>>>>>>> http://docs.wso2.org/wiki/display/Stratos200/WSO2+Stratos+Documentation >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Lakmal Warusawithana >>>>>>>> Software Architect; WSO2 Inc. >>>>>>>> Mobile : +94714289692 >>>>>>>> Blog : http://lakmalsview.blogspot.com/ >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> >>>>>>> *Sanjeewa Malalgoda >>>>>>> **B.Sc. Engineering(Hons) >>>>>>> Dip. in Com.Sc. >>>>>>> AMIESL , MIACSIT, CCNA >>>>>>> >>>>>>> * >>>>>>> Mobile +94713068779 >>>>>>> >>>>>>> http://sanjeewamalalgoda.blogspot.com/ >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Best Regards, >>>>>> Nirmal >>>>>> >>>>>> C.S.Nirmal J. Fernando >>>>>> Senior Software Engineer, >>>>>> WSO2 Inc. >>>>>> >>>>>> Blog: http://nirmalfdo.blogspot.com/ >>>>>> >>>>> >>>>> >>>>> Thanks, >>>>> -- >>>>> >>>>> *Sanjeewa Malalgoda** >>>>> >>>>> * >>>>> Mobile +94713068779 >>>>> >>>>> http://sanjeewamalalgoda.blogspot.com/ >>>>> >>>> >>>> >>>> >>>> -- >>>> ................................................... >>>> Dharshana Kasun Warusavitharana >>>> >>>> >>>> >>>> >>> >>> >>> -- >>> Lakmal Warusawithana >>> Software Architect; WSO2 Inc. >>> Mobile : +94714289692 >>> Blog : http://lakmalsview.blogspot.com/ >>> >>> >> >> >> -- >> Sameera Jayasoma >> >> blog: http://sameera.adahas.org >> twitter: https://twitter.com/sameerajayasoma >> flickr: http://www.flickr.com/photos/sameera-jayasoma/ >> > > > > -- > Lakmal Warusawithana > Software Architect; WSO2 Inc. > Mobile : +94714289692 > Blog : http://lakmalsview.blogspot.com/ > > -- Thanks, Shariq. Phone: +94 777 202 225
