Please catchup with Imesh on this; we had some chats on this where unfortunately you were not around. A common view needs to be reached, and then documentation can follow.
From: Reka Thirunavukkarasu [mailto:r...@wso2.com] Sent: Thursday, March 19, 2015 6:31 AM To: Shaheedur Haque (shahhaqu) Cc: dev; Mariangela Hills Subject: Re: [Discuss] Cartridge definition doesn't need "maxInstanceLimit" property anymore Hi Shaheed, Please find my comments inline. Unfortunately i think that we don't currently have the documentation covering all the scenarios it seems..Will work with Mari to add the missing scenarios.. On Wed, Mar 18, 2015 at 3:51 AM, Shaheedur Haque (shahhaqu) <shahh...@cisco.com<mailto:shahh...@cisco.com>> wrote: Hi Reka, Thanks for the detailed explanation. See below… From: Reka Thirunavukkarasu [mailto:r...@wso2.com<mailto:r...@wso2.com>] Sent: Wednesday, March 18, 2015 5:08 AM To: dev Cc: Shaheedur Haque (shahhaqu) Subject: Re: [Discuss] Cartridge definition doesn't need "maxInstanceLimit" property anymore HI Shaheed, Let me explain on why we had to have partitionMax, cartridgeMin and cartridgeMax rather than just having a partitionMin and partitionMax. There are two new concepts in 4.1.0. 1. Cartridge group level deployment/Cluster level deployment policy I believe that you are aware of what is cartridge group from my earlier mail. In that case, we have introduced this cartridge group level deployment policy in order to have deployment pattern per group level. So that you can have a deployment policy per cartridge group. In that case, all the children of that cartridge group will inherit the parent's deployment pattern. It will be useful to achieve high availability for your group. When using such policy in group level, underlying cartridges will not need to have their own cartridge level policy. But cartridges still need to enforce what is the min/max members it is required. For that, we used cartridgeMin and cartridgeMax in the application. Eg: If you have C1, C2 part of G1. Then you have defined a GroupLevel policy for G1 to use P1 and P2 with RoundRobin algorithm. In that case, stratos will make sure that the one G1 instance creates in P1 and consecutive G1 next instance creates in P2. So that you can achieve HA as your GroupInstance level. [srh] I can see why one might want this. However, none of the documentation I have seen hints at this group level policy, and I’m afraid that without it, it is not possible for me to make any sense of the cartridgeMin/Max values. So, please provide the missing docs; for example, where is this group level RoundRobin (or whatever) setting applied. Sure..We will need to add this to documentation..Will provide contents for the scenarios which will cover the RoundRobin or One-after-another algorithm with GroupLevel Deployment policy. 2. As i explained earlier, we have group instances concept. So, we need to specify a min/max per associated cluster instance level rather than just specifying a global min as the partitionMin. That's why introduced cartridgeMin and cartridgeMax in the application. So that it can be applicable per cluster instance level. But we need to control the actual partition level maximum in order to avoid resource exhausted. Hence, we had to have the partitionMax as global max applicable for the cartridge level. So that our drools will not create members when the partitionMax is reached even though you have room in cartirdgeMax. Based on these two concepts, we had to have partitionMax, cartridgeMin and cartridgeMax. If we change these to just with partitionMin and partitionMax, then #1 concept might get broken. We will need to find a way to achieve #1. Not sure whether i could explain everything here. I have highlighted the concepts a bit according my understanding.. Please let me know your concerns on this.. [srh] I now understand the desire for a Group Policy. However, what I need to know now is the exact rules for how to use it, and how it interacts with the Deployment Policy. Specifically documentation is needed to cover all the possible cases: 1. What happens if I don’t want a Group Policy and only use the Deployment Policy? Is that allowed? How is that expressed? Yah..It is actually allowed in our implementation. That's upto the user to select whether to use GroupLevelPolicy/Cartridge level deployment policy. Without groupLevel policy, Deployment policy in the cartridge level can only be used to deploy the application. 2. What happens if I only have a Group Policy and no Deployment Policy? Is that allowed? How is that expressed? Yah..According to current design, if we specify Group Policy, then no need of Deployment policy any of the children of that group. In that case, in the cartirdge level we will need to find out how many members it is required. For that, we used to specify it through the cartirdgeMin/Max. If we remove cartirdgeMin/Max, we will need to think of a way to handle this.. 3. What happens if I have neither Group Policy nor Deployment Policy? Is that allowed? I don't think that it is possible. You should have either GroupPolicy in group level or Deployment Policy at the leaf leavl (for all the cartirdges) in the application in order to deploy it. 4. What happens if I have both a Group Policy and a Deployment Policy? In the current implementation, GroupPolicy will get the priority. So, Deployment Policy will get simply ignored. 5. In case 4, how exactly do the min and max sets of values interact? (Notice that in this thread, it had previously been asserted that the Group values could be defaulted from the Partition Values, is that still the plan?) What are all the edge cases, e.g. if the CartridgeMin is higher than the sum of the PartitionMaxs. Will that cause a validation error? Yah..I understood your point here..Having min/max in application and partitionMax in Deployment policy makes addition complication. We can go ahead with paritionMin/Max. In that way, with the cartridge level deployment policy, i don't see a problem in getting things working. But we would need to fix GroupLevelDeployment Policy. I'm also not exactly clear about how the group values are working now..Will check more on this min/max handling according to the current implementation and update you more on this.. In other words, I am asking for some indication that the exact behavior has been designed and should be well behaved ehough to consider exposing to my users. Thanks for bringing the points Shaheed.. These are really valid points in order to start with the documentation on GroupLevel/Cartirdge level deployment policy. Thanks, Reka Thanks, Shaheed Thanks, Reka On Tue, Mar 17, 2015 at 9:13 PM, Imesh Gunaratne <im...@apache.org<mailto:im...@apache.org>> wrote: Hi Shaheed, We are sorry to hear the problems that you have encountered while trying to use the latest codebase. Yes I agree that there are bits and pieces missing in the documentation at the moment. Shall we have a quick call to get things sorted? Anyone interested can also join. Thanks On Tue, Mar 17, 2015 at 2:52 PM, Lahiru Sandaruwan <lahi...@wso2.com<mailto:lahi...@wso2.com>> wrote: Hi, Sorry for the typos. Agreed with Shaheed on Max values. We only have one place to define minimum already, which is cartridge min. We can move that as partitionMin and find the Cartridge min using them. Thanks. On Tue, Mar 17, 2015 at 2:37 PM, Rajkumar Rajaratnam <rajkum...@wso2.com<mailto:rajkum...@wso2.com>> wrote: Hi, I agree with Shaheed. cartridgeMax is the addition of all partitions' partiionMax values and cartridgeMin is the addition of all partition's partitioinMin values in the deployment policy that the cartridge is referring to. As Lakmal mentioned, we can calculate cartridgeMax/Min using partitionMax/Min in deployment policy. Thanks. On Tue, Mar 17, 2015 at 2:22 PM, Lakmal Warusawithana <lak...@wso2.com<mailto:lak...@wso2.com>> wrote: Hi, I think, we only need two values min/max. Min is need to get dependency ratio (and HA) and max is need for one-after-other deployment patten or absolute max. IMO, if we can set all these values in deployment policy (with respect to partitions) would be very clear. Only reason I can see, put CartridgeMin in application is to calculate dependency ratio. Can we calculate it from attached deployment policy? On Tue, Mar 17, 2015 at 1:50 PM, Shaheedur Haque (shahhaqu) <shahh...@cisco.com<mailto:shahh...@cisco.com>> wrote: LOL. We are going around in circles…let’s start over. In 4.1.0, there are two pairs of limits on instance numbers, one called partitionMin/Max (specified in the Deployment Policy) and one called cartridgeMin/Max alongside the subscription info (specified in the Application). My question was what is the point of cartridgeMax? You gave a very confusing example in reply with what look like several crucial typos, and even if I guess at what was intended, I still don’t see a strong reason to have cartridgeMax. Specifically, in your example, if I wanted to limit the application to 5 instances of PHP, why would I not simply set the partitionMax to 2,3 or 3,2 in the two partitions? Similarly, I have yet to see any convincing response as to why cartridgeMin is really required. The subtle shift in behavior being claimed in these cases seems confusing at best, and simply broken at worst. I claim broken because there are no clear rules as to how all 4 values relate, and no clear use-case to have all 4. In short, I have no idea how or why I should set them. Please clarify or remove. Thanks, Shaheed From: Lahiru Sandaruwan [mailto:lahi...@wso2.com<mailto:lahi...@wso2.com>] Sent: Tuesday, March 17, 2015 3:43 AM To: dev Subject: Re: [Discuss] Cartridge definition doesn't need "maxInstanceLimit" property anymore Hi Shaheed, We can define the maximum number of instances that can be spawned per partition in deployment policy. You can find a simple sample which is written for next developer preview at [1]. This sample is an end to end sample with single cartridge application and Mock IaaS. See the step 5 for partitionMax usage. Thanks. [1] https://cwiki.apache.org/confluence/display/STRATOS/4.1.0-Beta+Install+Stratos+with+a+Mock+IaaS On Mon, Mar 16, 2015 at 5:24 PM, Shaheedur Haque (shahhaqu) <shahh...@cisco.com<mailto:shahh...@cisco.com>> wrote: Hi Lahiru, What do you mean by partition max? Where is it specified? Thanks, Shaheed From: Lahiru Sandaruwan [mailto:lahi...@wso2.com<mailto:lahi...@wso2.com>] Sent: Saturday, March 14, 2015 2:31 AM To: dev Subject: Re: [Discuss] Cartridge definition doesn't need "maxInstanceLimit" property anymore Hi Shaheedur, This is the same model we had from 4.0.0 and we did not change the concept in this release. Explanation as below, There can be several partitions that a particular cartridge(service cluster) can span over. E.g. Say PHP cartridge have 3 partitions(P1 and P2). P1 has 3 as partition max and P1 has 6 as partition max. PHP has 2 as Cartridge min and 5 as Cartridge max. With one-after-another algorithm between partitions, Min instances creation, Both the minimum instances will be created in P1. Scaling up, First scaling up member will be created in P1 and the next members will be created in P2. It can only create 3 members in P1 and 2 members in P2 as it will reach it's max of 5 members. That's where the cartridge max is used. With round-robin algorithm between partitions, Min instances creation, One of the minimum instances will be created in P1 and the other in P2. Scaling up, Scaling up members will be created in round-robin manner in P1 and P2. On Fri, Mar 13, 2015 at 9:31 PM, Shaheedur Haque (shahhaqu) <shahh...@cisco.com<mailto:shahh...@cisco.com>> wrote: After discussing a bit with Imesh, we identified 3 points that need clarifying: • If there is to be a cartridgeMin attribute, it should at least be optional and take its default value from the deployment policy minimum value. Do you mean the minimum value per partition? I cannot see such value. There is only maximum in partitions of deployment policy. • If there is to be a cartridgeMin attribute, its behaviour needs to be specified with respect to the deployment min and max. For example, what happens if the cartridgeMin is greater than the deployment max? This is a good question. Ideally, with current implementation, it will not create instances beyond partition Max. • We need to find out what, if anything, the cartridgeMax is for. IMO partition max are for limiting the instances in IaaS, for particular region, zone or so. But cartridge need it's own limit when it scales up, where reusable partitions may have less or more space. Therefore we need both. Stratos will create instances until the lowest of those max values(Cartridge max and partition max). Thanks. From: Shaheedur Haque (shahhaqu) Sent: 13 March 2015 08:22 To: dev@stratos.apache.org<mailto:dev@stratos.apache.org> Subject: RE: [Discuss] Cartridge definition doesn't need "maxInstanceLimit" property anymore Hi Reka, First, I strongly suggest we not use internal terms like “cluster”, since they are not part of the user visible model, to explain user visible behaviour. For example, my understanding of a cluster is almost certainly not as good as you think it is ☺. Nevertheless… I think I understand: you are saying that when a group scales up, you will initially spin up cartridgeMin instances, correct? If this is not correct, please clarify. If it is correct, then: • I would have thought that the correct behaviour was to enforce the minimum value specified in the deployment policy. • Even if there is a theoretical use case where a new group should start with a different values that the one in the deployment policy, you would need to clearly explain how cartridgeMin relates to both deployment min *and* max when the values clash. I say we should rather keep it simple and use the deployment policy values. • What is cartridgeMax used for? Thanks, Shaheed From: Reka Thirunavukkarasu [mailto:r...@wso2.com] Sent: 13 March 2015 02:54 To: dev Subject: Re: [Discuss] Cartridge definition doesn't need "maxInstanceLimit" property anymore Hi Shaheedur, Sorry for the confusion..Let me explain what cartridgeMin is and the purpose of having it in the application. As you already aware, we have the group instances/cluster instances concept with 4.1.0 in order to support group scaling. For the cluster level, we will need a minimum count for the members in order to maintain this minimum count all the time. Since we have cluster instance concept, we will need a minimum members per cluster instance level. So that whenever a new cluster instance is getting created, we can satisfy the cluster instance by creating the cartridgeMin number of members and can send the clusterInstanceActivated event. That's why cartridgeMin got introduced in the application. "cartridgeMin" means the minimum number of members per cluster instance. When a cluster instance is getting created, let's say you have two as the cartridgeMin. In that case in order to create those two members, we will need to find out the partition. For that, we will get the associated policy and find the suitable partition to spin those members one by one. If one of the partition is full, our algorithm is capable of choosing the next available partition. Please let me know, if it is unclear still.. Thanks, Reka On Thu, Mar 12, 2015 at 11:31 AM, Shaheedur Haque (shahhaqu) <shahh...@cisco.com<mailto:shahh...@cisco.com>> wrote: I’m now thoroughly confused as to what is going and staying. I *think* the latest part of this thread says we need to keep cartridgeMin and cartridgeMax. Why do we need cartridgeMin and cartridgeMax at this point in the system? From: Reka Thirunavukkarasu [mailto:r...@wso2.com<mailto:r...@wso2.com>] Sent: 12 March 2015 17:09 To: dev Subject: Re: [Discuss] Cartridge definition doesn't need "maxInstanceLimit" property anymore Hi Raj, On Thu, Mar 12, 2015 at 10:01 AM, Rajkumar Rajaratnam <rajkum...@wso2.com<mailto:rajkum...@wso2.com>> wrote: On Thu, Mar 12, 2015 at 10:24 PM, Reka Thirunavukkarasu <r...@wso2.com<mailto:r...@wso2.com>> wrote: Hi Lahiru/Raj, I think that we have introduced this cartridgeMin/cartridgeMax when introducing the deployment policy for cartridge and groups as global deployment policy. Since we have the same concept now, i would like to review the implementation and confirm whether it is required or not. Can you please hold until that? I will quickly confirm on this.. Hi Reka, I am not sure whether I understood you wrong. Yes we need cartridgeMin and cartridgeMax, which are defined in application json. But we don't need maxInstanceLimit property, which used to define in cartridge json in 4.0.0. I guess we have already removed references to this property from code base, but not from sample artifacts. Thanks for the details.. We no longer using maxInstanceLimit. +1 to remove it..I was bit confused when i read the mail body as i thought that you are going to remove cartidgeMin/cartridgeMax. Now it is clear.. Thanks, Reka Thanks, Reka On Thu, Mar 12, 2015 at 9:40 AM, Lahiru Sandaruwan <lahi...@wso2.com<mailto:lahi...@wso2.com>> wrote: Noticed today. It was misleading as we have this left in samples. I will clean this up. Thanks. On Thu, Mar 12, 2015 at 10:04 PM, Rajkumar Rajaratnam <rajkum...@wso2.com<mailto:rajkum...@wso2.com>> wrote: Hi Devs, We are defining cartridge min max count in application definition. { "applicationId": "single-cartridge-app", "alias": "single-cartridge-app", "multiTenant": false, "components": { "cartridges": [ { "type": "php", "cartridgeMin": 1, "cartridgeMax": 10, "subscribableInfo": { "alias": "my-php", "autoscalingPolicy": "autoscaling-policy-1", "deploymentPolicy": "deployment-policy-1", "artifactRepository": { "privateRepo": false, "repoUrl": "https://github.com/imesh/stratos-php-applications.git";, "repoUsername": "", "repoPassword": "" } } } ] } } In 4.0.0, we used to define these in cartridge definition, in IaaS provider section. We have now removed it from cartridge bean classes. However I can see that samples still have this attribute. I will remove it. Thanks. -- Rajkumar Rajaratnam Committer & PMC Member, Apache Stratos Software Engineer, WSO2 Mobile : +94777568639<tel:%2B94777568639> Blog : rajkumarr.com<http://rajkumarr.com> -- -- Lahiru Sandaruwan Committer and PMC member, Apache Stratos, Senior Software Engineer, WSO2 Inc., http://wso2.com lean.enterprise.middleware phone: +94773325954<tel:%2B94773325954> email: lahi...@wso2.com<mailto:lahi...@wso2.com> blog: http://lahiruwrites.blogspot.com/ linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146 -- Reka Thirunavukkarasu Senior Software Engineer, WSO2, Inc.:http://wso2.com, Mobile: +94776442007<tel:%2B94776442007> -- Rajkumar Rajaratnam Committer & PMC Member, Apache Stratos Software Engineer, WSO2 Mobile : +94777568639<tel:%2B94777568639> Blog : rajkumarr.com<http://rajkumarr.com> -- Reka Thirunavukkarasu Senior Software Engineer, WSO2, Inc.:http://wso2.com, Mobile: +94776442007<tel:%2B94776442007> -- Reka Thirunavukkarasu Senior Software Engineer, WSO2, Inc.:http://wso2.com, Mobile: +94776442007<tel:%2B94776442007> -- -- Lahiru Sandaruwan Committer and PMC member, Apache Stratos, Senior Software Engineer, WSO2 Inc., http://wso2.com lean.enterprise.middleware phone: +94773325954<tel:%2B94773325954> email: lahi...@wso2.com<mailto:lahi...@wso2.com> blog: http://lahiruwrites.blogspot.com/ linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146 -- -- Lahiru Sandaruwan Committer and PMC member, Apache Stratos, Senior Software Engineer, WSO2 Inc., http://wso2.com lean.enterprise.middleware phone: +94773325954<tel:%2B94773325954> email: lahi...@wso2.com<mailto:lahi...@wso2.com> blog: http://lahiruwrites.blogspot.com/ linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146 -- Lakmal Warusawithana Vice President, Apache Stratos Director - Cloud Architecture; WSO2 Inc. Mobile : +94714289692<tel:%2B94714289692> Blog : http://lakmalsview.blogspot.com/ -- Rajkumar Rajaratnam Committer & PMC Member, Apache Stratos Software Engineer, WSO2 Mobile : +94777568639<tel:%2B94777568639> Blog : rajkumarr.com<http://rajkumarr.com> -- -- Lahiru Sandaruwan Committer and PMC member, Apache Stratos, Senior Software Engineer, WSO2 Inc., http://wso2.com lean.enterprise.middleware phone: +94773325954<tel:%2B94773325954> email: lahi...@wso2.com<mailto:lahi...@wso2.com> blog: http://lahiruwrites.blogspot.com/ linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146 -- Imesh Gunaratne Technical Lead, WSO2 Committer & PMC Member, Apache Stratos -- Reka Thirunavukkarasu Senior Software Engineer, WSO2, Inc.:http://wso2.com, Mobile: +94776442007<tel:%2B94776442007> -- Reka Thirunavukkarasu Senior Software Engineer, WSO2, Inc.:http://wso2.com, Mobile: +94776442007<tel:%2B94776442007>
