Hi, +1 for the improvement to scalability. So in this improvement, we use a thread per partition when ever we run the minimum check.
But i think the best way is to use a thread per spawning/ terminating call to Cloud Controller, and keeping the minimum rule per cluster. Also some partitions can have Zero as the minimum(Most probably a higher percentage) and running a thread for checking that is costly. This feature will be required when ever we do a spawning or terminating(In scale up/down with scale up/down rule). So IMO we should not mix up using a thread for the blocking call and running the rule. Created [1] to track this. [1] https://issues.apache.org/jira/browse/STRATOS-253 Thanks. On Tue, Dec 3, 2013 at 1:46 PM, Nirmal Fernando <[email protected]>wrote: > Here's the correct rule file as of now. I'll be keep adding other > functions. > > global java.util.Map partitionCtxts; > global java.lang.String clusterId; > > import function > org.apache.stratos.autoscaler.rule.AutoscalerRuleEvaluator.delegateSpawn; > import function > org.apache.stratos.autoscaler.rule.AutoscalerRuleEvaluator.delegateTerminate; > > > rule "Minimum Rule" > dialect "mvel" > when > $ctxt : PartitionContext () > eval($ctxt.getCurrentMemberCount() < > $ctxt.getMinimumMemberCount()) > > then > memberContext = delegateSpawn($ctxt.getPartition(),clusterId); > if(memberContext != null) { > $ctxt.addPendingMember(memberContext); > } > > end > > rule "Terminate Obsoleted Instances" > > dialect "mvel" > when > $ctxt : PartitionContext () > eval($ctxt.getObsoletedMembers().size() > 0) > memberId : String() from $ctxt.getObsoletedMembers() > eval($ctxt.removeObsoleteMember(memberId)) > then > delegateTerminate(memberId); > end > > > > > On Tue, Dec 3, 2013 at 9:06 AM, Nirmal Fernando <[email protected]>wrote: > >> >> >> >> On Tue, Dec 3, 2013 at 8:13 AM, Lahiru Sandaruwan <[email protected]>wrote: >> >>> Thanks Nirmal and Udara. >>> >>> Just found "minimum-autoscaler-rule.drl" at tests of autoscaler >>> component. >>> >>> It seems this rule([1]) is run against a partition. IMO we should run it >>> against a cluster since rules can be run in loops efficiently as follows, >>> >> >> That doesn't scale, that's the whole point of the refactoring effort I've >> put, last few days. >> >> The idea is, cluster monitor monitors all of its partitions and make sure >> that the minimum instances of that Partition is there, all the time. This >> job is an independent one from partition to partition, hence the idea is to >> run this minimum check parallel. If you embed this in a for loop inside the >> rule, it's sequential processing. >> >>> >>> rule "Minimum Rule" >>> dialect "mvel" >>> when >>> >>> $deploymentPolicy : $cluster.getDeploymentPolicy() >>> $partitionGroup : PartitionGroup () from >>> $deploymentPolicy.getPartitionGroups() >>> $partition : Partition () from $partitionGroup.getPartitions() >>> eval($clusterContext.getMemberCount($partition.getId()) < >>> $partition.getPartitionMembersMin()) >>> >>> then >>> int memberCountToBeIncreased = 1 ; >>> if(AutoscalerRuleEvaluator.delegateSpawn($ctxt.getPartition() >>> ,$clusterId)){ >>> $ctxt.incrementCurrentMemberCount(memberCountToBeIncreased); >>> } >>> >>> end >>> >>> Thanks. >>> >>> [1] >>> rule "Minimum Rule" >>> dialect "mvel" >>> when >>> $ctxt : PartitionContext () >>> eval($ctxt.getCurrentMemberCount() < >>> $ctxt.getMinimumMemberCount()) >>> >>> then >>> int memberCountToBeIncreased = 1 ; >>> if(AutoscalerRuleEvaluator.delegateSpawn($ctxt.getPartition() >>> ,$clusterId)){ >>> $ctxt.incrementCurrentMemberCount(memberCountToBeIncreased); >>> } >>> >>> end >>> >>> >>> On Tue, Dec 3, 2013 at 7:29 AM, Udara Liyanage <[email protected]> wrote: >>> >>>> The min rule file us updated as I can remember. It is not the one you >>>> have specified in the mail. Not sure updated rule file is added to the >>>> product or not. >>>> >>>> Touched, not typed. Erroneous words are a feature, not a typo. >>>> On Dec 2, 2013 11:42 PM, "Lahiru Sandaruwan" <[email protected]> wrote: >>>> >>>>> >>>>> >>>>> >>>>> On Mon, Dec 2, 2013 at 11:35 PM, Lahiru Sandaruwan >>>>> <[email protected]>wrote: >>>>> >>>>>> Nirmal, >>>>>> >>>>>> In minimum rule, it runs against all the clusters[1]. But in >>>>>> ClusterMonitor thread is run per cluster which is correct. >>>>>> >>>>>> Didn't you update "Minimum Rule" in the rules file, or is it a >>>>>> separate rules file that you have used? >>>>>> >>>>>> >>>>> I meant whether you have used a different Rules file at deployment >>>>> time of the Stratos, which is different to default one in the >>>>> products/autoscaler at [1]. >>>>> >>>>> Thanks. >>>>> >>>>> >>>>> >>>>>> >>>>>> [1] >>>>>> >>>>>> rule "Minimum Rule" >>>>>> dialect "mvel" >>>>>> when >>>>>> $service : Service () >>>>>> $cluster : Cluster () from $service.getClusters() >>>>>> $deploymentPolicy : DeploymentPolicy(id == >>>>>> $cluster.getDeploymentPolicyName() ) from >>>>>> $manager.getDeploymentPolicyList() >>>>>> $partitionGroup : PartitionGroup () from >>>>>> $deploymentPolicy.getPartitionGroups() >>>>>> $partition : Partition () from $partitionGroup.getPartitions() >>>>>> $clusterContext : ClusterContext() from >>>>>> $context.getClusterContext($cluster.getClusterId()) >>>>>> eval($clusterContext.getMemberCount($partition.getId()) < >>>>>> $partition.getPartitionMembersMin()) >>>>>> >>>>>> >>>>>> then >>>>>> int memberCountToBeIncreased = >>>>>> $partition.getPartitionMembersMin() - >>>>>> $clusterContext.getMemberCount($partition.getId()); >>>>>> >>>>>> if($evaluator.delegateSpawn($partition,$cluster.getClusterId(), >>>>>> memberCountToBeIncreased)){ >>>>>> >>>>>> $clusterContext.increaseMemberCountInPartitionBy($partition.getId(), >>>>>> memberCountToBeIncreased); >>>>>> } >>>>>> end >>>>>> -- >>>>>> -- >>>>>> Lahiru Sandaruwan >>>>>> Software Engineer, >>>>>> Platform Technologies, >>>>>> WSO2 Inc., http://wso2.com >>>>>> lean.enterprise.middleware >>>>>> >>>>>> email: [email protected] cell: (+94) 773 325 954 >>>>>> blog: http://lahiruwrites.blogspot.com/ >>>>>> twitter: http://twitter.com/lahirus >>>>>> linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146 >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> -- >>>>> Lahiru Sandaruwan >>>>> Software Engineer, >>>>> Platform Technologies, >>>>> WSO2 Inc., http://wso2.com >>>>> lean.enterprise.middleware >>>>> >>>>> email: [email protected] cell: (+94) 773 325 954 >>>>> blog: http://lahiruwrites.blogspot.com/ >>>>> twitter: http://twitter.com/lahirus >>>>> linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146 >>>>> >>>>> >>> >>> >>> -- >>> -- >>> Lahiru Sandaruwan >>> Software Engineer, >>> Platform Technologies, >>> WSO2 Inc., http://wso2.com >>> lean.enterprise.middleware >>> >>> email: [email protected] cell: (+94) 773 325 954 >>> blog: http://lahiruwrites.blogspot.com/ >>> twitter: http://twitter.com/lahirus >>> linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146 >>> >>> >> >> >> -- >> Best Regards, >> Nirmal >> >> Nirmal Fernando. >> PPMC Member & Committer of Apache Stratos, >> Senior Software Engineer, WSO2 Inc. >> >> Blog: http://nirmalfdo.blogspot.com/ >> > > > > -- > Best Regards, > Nirmal > > Nirmal Fernando. > PPMC Member & Committer of Apache Stratos, > Senior Software Engineer, WSO2 Inc. > > Blog: http://nirmalfdo.blogspot.com/ > -- -- Lahiru Sandaruwan Software Engineer, Platform Technologies, WSO2 Inc., http://wso2.com lean.enterprise.middleware email: [email protected] cell: (+94) 773 325 954 blog: http://lahiruwrites.blogspot.com/ twitter: http://twitter.com/lahirus linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
