Hi Martin, Please find my comments inside related to this group scaling implementation.
> *Subject:* [Discuss][Grouping] Handling Group level scaling in Composite
> Application
>
>
>
> Hi
>
>
>
> This is to discuss $subject. In the case of scaling with composite
> application, we can divide it into three parts as Martin has also explained
> in previous mails. I summarise as below from a Martin's mail with the
> proposed possible solution in stratos to support group level scaling:
>
>
>
> o scaling by statistics,
>
> o scaling by group member and
>
> o scaling by group.
>
>
>
> Based on this, the general algorithm would be like this (in order):
>
> 1. Scale VMs until the cluster maximum is reached (in at least one
> of the cluster within the group - scale by statistics)
>
> We can address this in stratos with the usual autoscaling based on
> statistics. A single cluster monitor is capable of taking this decision to
> scale its own members.
>
> 2. Scale up a new cluster of same type as the one which has reached
> the maximum of VMs until the max member number is reached (scale by group
> member).
>
> If a cluster of a group reaches the max based on the deployment policy,
> then if there is a room in the partition to spin more instances, we can
> simply update the deployment policy of that cluster to increase the max
> instances. If there are more than one cluster of the group resides in a
> partition, we can divide the max instances of the partition among all of
> those clusters by keeping a ratio among those clusters like 3C1:4C2. In
> case, when the partition is max out, if we extend the partition with more
> hardware, then again we can update the deployment policy with new
> max values of those clusters. So that relevant cluster monitors will
> execute with the updated values.
>
> “Martin:” is the max instance adjusted automatically by the autoscaler or
> does it have to be done per “user” request ?
>
Yah. In this case, the max can be adjusted manually using the manual
scaling support in stratos now. I'm not sure whether autoscaler can adjust
this max automatically.
> 3. Scale up a new group instance of the same group type (or
> definition), including all the respective dependencies (scale by group)
>
> We can achieve this by using combination of round-robin and
> one-after-another algorithm in the deployment policy. For Eg:
>
> You can deploy a group(G1) which contains of C1 and C2 clusters in
> partition P1 and P2 using round-robin algorithm. So that C1 and C1 will get
> the high availability. You can have another idle partition called P3. When
> you decided to scale by group, then using one-after-another algorithm in
> deployment policy, we can choose the P3 to bring up the G1 with relevant
> minimum instances.
>
> In that way, we can improve our deployment policy to support combination
> of these algorithms within a network partition. When we have P1, P2 and P3
> in a network partition, we will be able to use round-robin among P1 and P2
> and we can use one-after-another among (P1, P2) and P3.
>
> “Martin:” I am not entirely sure I completely understand how this works
> using the round robin algorithm and the partitioning, I think it would be
> helpful to demonstrate the algorithm by using a more complex group with
> multiple cartridges and nested sub groups, including the group id /
> subscription alias generation, and event generation ?
>
I have attached here with a sample json which would explain you about this
new partition groups and algorithm. The deployment Policy can have the
following:
DeploymentPolicy
+NetworkPartitions
+ id
+ partitionGroupAlgo(will be applicable between partition
groups)
+ partitionGroups
+ id
+ partitionAlgo(will be applicable between partitions)
+ patitions
As per the attached policy, autoscaler will choose the
p1-p2-group partition group for the initial cluster monitor to start
instances. When that p1-p2-group got to maxed out, it can notify the parent
and choose the p3-p4-group for further spinning instances. When group gets
the notification, it can notify other dependent child to switch to another
partitionGroup. So, the dependent cluster will choose another defined
partitionGroup with one-after-another algorithm. The requirement here is
that both dependent clusters have to have the same number of
partitionGroups available.
> A few more questions:
> - when we spin up new instances of a group using the above mentioned
> algorithm will it also subsequently scale all the respective dependencies ?
> One of the main advantages of grouping (IMHO)) is that it scales up / down
> not only a specific instance of a group but also subsequent dependencies
> (cartridges and nested groups).
>
Yah..As i explained earlier the notification to the parent group will
handle this.
> - For the new group instances, how will the group Id differ from the group
> which is scaled, how would we generate group Ids ?
>
Since we use this algorithm, we no longer need a new group id to be
generated to handle this.
> - How will the scale down work (or termination of a group and it’s
> dependencies) ?
>
Scale down will also be handled by this algorithm. According to the
attached definition, p1-p2-group got maxed out and we chose p3-p4-group
using one-after-another, then until the chosen p3-p4-group got wiped out,
we won't be scaling down the instances in p1-p2-group.
Please let me know, if you need further clarification on this....
Thanks,
Reka
> - What about group events, which event will be generated (e.g. group
> active, group down, etc and what parameters like group name, group id, app
> id ?
>
> Please share your thoughts on the above approach and Please add, if i
> have missed any other points here..
>
>
>
> Thanks,
>
> Reka
>
>
>
> --
>
> Reka Thirunavukkarasu
> Senior Software Engineer,
> WSO2, Inc.:http://wso2.com,
>
> Mobile: +94776442007
>
>
>
--
Reka Thirunavukkarasu
Senior Software Engineer,
WSO2, Inc.:http://wso2.com,
Mobile: +94776442007
Deployment-Policy-Grouping.json
Description: application/json
