Hi Martin, Can you please explain more about how you subscribed to a large number of cartridges? AFAIU, you have subscribed to them one after the other. Please confirm.
What you have observed is correct. Stratos would spin up cartridges for each subscriptions sequentially. We did not have a use case for spinning up instances is parallel since in Stratos 4.0.0 there were no concept of a subscribing at once to more than one cartridge; whether they should be spinning up one after the other (dependencies) or if they should be spinning in parallel (independent). AFAIU we address this in Service Grouping, which is currently happening. On Mon, Sep 29, 2014 at 6:51 AM, Martin Eppel (meppel) <mep...@cisco.com> wrote: > > > We are observing an interesting phenomena (based on stratos 4.0.0), when > we subscribe a large number of cartridges (40+) it takes up to 40+ minutes > for the VMs to spin up. From the logs we can observe the publishing > subscriptions goes fairly quickly but VM spin up takes up quite a bit of > time. For example, if we publish a few new subscriptions on top of already > running VMs, publishing the subscriptions take 10+ seconds, while spinning > up the corresponding VMs take minutes. > > > > Based on this observation, it appears that stratos is spinning up VMs not > in parallel but rather sequentially even across different subscriptions > (one VM at a time). > > > > However, my understanding from analyzing the code is that it should happen > asynchronously instead of sequentially. Is there something in stratos wich > serializes the spin up of VM (across different sbscriptions) ? > > > > Below is mu analysis on how subscriptions and Vm spin up works in stratos, > is this correct, or did I miss something? If yes, what could potentially > cause the appearantly sequential spin up of the VMs ? > > > > > > Thanks > > > > Martin > > > > > > Stratos subscription and VM spin up: > > In general, cartridge subscription and VM spin up is not serialized (at > least for different subscriptions), it’s all event driven and for each > cartridge a separate monitoring thread is created. However, spinning up VMs > is rule driven (per cluster) in the autscaler and is determined by the > setting of the min number of VMs in a cluster as well as the health > statistics for scaling (which are reported back by the cartridge agent and > averaged by the autoscaler). The rule is checked periodically, so it does > take time for a VM to spin up. > > > > For the rule to kick in the Cluster has to be in a certain state, and for > the cluster state to be set it again depends on an event to be received so > I think there is a good chance that when the rule starts up the first time > (as [part of the periodic checks in the ClusterMonitor) it has to wait for > a subsequent run for the Cluster to be in the right state. > > > > In one aspect it does seem to be serialized, for each Cluster (== > subscription) to reach the min number of VMs, VMs are spawned sequentially > for each periodic check (it does not seem to spawn n number of VMs at the > same time). > > -- > Thanks and Regards, > > Isuru H. > +94 716 358 048* <http://wso2.com/>* > > > * <http://wso2.com/>* > > >
