Thanks Martin for the changes... On Thu, Oct 9, 2014 at 6:27 AM, Martin Eppel (meppel) <mep...@cisco.com> wrote:
> Hi Reka, > > > > I adjusted the remaining code to the new scheme and also modified the code > in the DependencyBuilder (and checked it in) but I am not sure if it is > correct, again, since I am not clear on the use of “*private List<String> > startList;*” , please take a look and let me know how it is supposed to > work. > > > Also, there is still an issue the the startupOrder not being properly set > in the DependencyBuilder, will look into it tomorrow, > Sure..I'm in the process of testing the flow..Will update on it.. > > > Thanks > > > > Martin > > > > *From:* Martin Eppel (meppel) > *Sent:* Wednesday, October 08, 2014 12:14 PM > *To:* Reka Thirunavukkarasu > *Cc:* Isuru Haththotuwa (isu...@wso2.com); dev@stratos.apache.org > *Subject:* DependencyBuilder / StartupOrder clarification > > > > Hi Reka, > > > > I need some clarification on how the DependencyBuilder is supposed to work > (while I am trying to replace the code with the new scheme to represent the > startup order). > > > > In the class StartupOrder.java we maintain a structure “*private > List<String> startList;*” which is being used later in the “ > *DependencyBuilder.java*”, see code snipplet below. However, I can’t find > anywhere in the code that the “startList” is initialized or set > (getStartList(…) being invoked). I am not really sure what the intention is > how this code is supposed to work ? > > > > Thanks > > > > Martin > > > > DependencyBuilder.java > > > > *Set<StartupOrder> startupOrderSet = dependencyOrder.getStartupOrders();* > > * ApplicationContext foundContext = null;* > > * for (StartupOrder startupOrder : startupOrderSet) {* > > * foundContext = null;* > > * for (String start : startupOrder.getStartList()) {* > > * ApplicationContext applicationContext = > ApplicationContextFactory.* > > * getApplicationContext(start, > component, dependencyTree);* > > * String id = applicationContext.getId(); //TODO change > the id* > > > > * ApplicationContext existingApplicationContext =* > > * > dependencyTree.findApplicationContextWithId(id);* > > * if (existingApplicationContext == null) {* > > * if (foundContext != null) {* > > * //appending the start up order to existing > group/cluster* > > * > foundContext.addApplicationContext(applicationContext);* > > * if (log.isDebugEnabled()) {* > > * log.debug("Found an existing [dependency] > " + foundContext.getId() +* > > * " and adding the [dependency] " + > id + " as the child");* > > * }* > > * } else {* > > * //adding list of startup order to the > dependency tree* > > * > dependencyTree.addApplicationContext(applicationContext);* > > * }* > > * } else {* > > * if (foundContext == null) {* > > * //assigning the found context to the later > use.* > > * foundContext = existingApplicationContext;* > > * if (log.isDebugEnabled()) {* > > * log.debug("Found an existing [dependency] > " + id + " and setting it " +* > > * "for the next dependency to > follow");* > > * }* > > * } else {* > > * //TODO Throw exception, since another same > start order already found* > > * log.warn("Startup order is not consistent. It > contains the group/cluster " +* > > * "which has been used more than one in > another startup order");* > > * }* > > > > * }* > > > > * }* > > > > * }* > > > > *From:* Reka Thirunavukkarasu [mailto:r...@wso2.com <r...@wso2.com>] > *Sent:* Wednesday, October 08, 2014 9:00 AM > *To:* Martin Eppel (meppel) > *Cc:* Isuru Haththotuwa (isu...@wso2.com); dev@stratos.apache.org > *Subject:* Re: [Grouping][Part-2] Composite Application Dependency Tree > building based in Autoscaler > > > > Hi Martin, > > > > On Wed, Oct 8, 2014 at 3:56 AM, Martin Eppel (meppel) <mep...@cisco.com> > wrote: > > Hi Reka, > > > > Just wanted to clarify, the changes suggested below should completely > replace the previous structure (StartupOrder with before, after), not only > in the json definition but also in all subsequent object models, correct ? > > Yah..We need to change the json, application parser and the Topology. > > > > Btw, what triggered this change ? > > As i think, it is naming issue. If we change it to terminate, then it > will be more consistent with the naming that we currently use in cloud > controller. Or did you mean something else? > > > > Thanks, > > Reka > > > > Thanks > > > > Martin > > > > *From:* Reka Thirunavukkarasu [mailto:r...@wso2.com] > *Sent:* Monday, October 06, 2014 2:22 AM > *To:* dev > *Subject:* Re: [Grouping][Part-2] Composite Application Dependency Tree > building based in Autoscaler > > > > Hi Imesh, > > > > On Mon, Oct 6, 2014 at 2:45 PM, Imesh Gunaratne <im...@apache.org> wrote: > > Hi Reka, > > > > I have a small concern on using the term "kill" in this scenario, I think > it would be much more elegant if we call it something like "terminate". > WDYT? > > +1. It's more appropriate to use terminate. Will change it to terminate. > > > > Thanks, > > Reka > > > > > > > > On Mon, Oct 6, 2014 at 11:44 AM, Reka Thirunavukkarasu <r...@wso2.com> > wrote: > > Hi all, > > > > I have implemented the dependency tree as mentioned in my mail earlier. It > will return the immediate children for the start able dependencies. > > > > FYI: a composite application has postgresGroup, php, mysqlGroup, app > server and esb as it's immediate children and their start up order is as > mentioned in the mail earlier. > > > > "startupOrders": [ > “postgresGroup, php", > "sqlGroup, tomcat", > "tomcat, apimanager", > "tomcat, esb” > ] > > > > So, if we look at the kill behaviour of this composite Application, it > will be like below: > > > > *kill-none* : none of them will be returned > > > > *kill-all*: all the elements in that dependency tree will be returned > > For eg: if something happened to postgresGroup, then all the children > of dependency tree would be returned as php, mysqlGroup, app server and esb > will be get killed. > > > > *kill-dependent*: all the children of that particular node in the > dependency tree will be returned. > > For eg: If something happened to mysqlGroup, then subsequently tomcat, > app server and esb would be get killed. > > > > Question: in case when we get more than one dependencies to be killed, can > we kill all of them in parallel or do we have to wait until it's dependent > cluster/group got killed? > > > > Thanks, > > Reka > > > > On Thu, Oct 2, 2014 at 5:58 PM, Reka Thirunavukkarasu <r...@wso2.com> > wrote: > > Hi Martin, > > > > On Wed, Oct 1, 2014 at 11:03 PM, Martin Eppel (meppel) <mep...@cisco.com> > wrote: > > Hi Reka, > > > > Are you suggesting to replace the current startupOrder definition with the > one mentioned below ? > > > > "startupOrder" : [ > > { > > "start":"aa", > > "after":"bb" > > } > > ] > > > > Replaced with > > > > "startupOrders": [ > > "mypostgresGroup, myphp", > > "mysqlGroup, mytomcat", > > "mytomcat, myapimanager", > > "mytomcat, myesb" > > ] > > > > I have a couple of questions, > > > > 1. If we use the cartridge alias and the group alias in the group / > application dependency definition how will it work when we auto scale > groups ? My current understanding is that to get group scaling to work we > would need 2 parameters – group name (==group.name) and group instance id > (== group.alias), one static and one dynamic. So I would think we’ll have > to define the application dependencies and group dependencies based on the > name and not the alias, but, during run time we have to calculate the > dependencies based on the alias. > > I think is important to make the distinction between group type (or name,) > and group instance Id, without it we won’t be able to implement group > scaling, wdyt ? > > Thanks for pointing this out..Yah..As you have mentioned, if we are to > scale the groups by creating new groups, then we will be unable to use the > groups alias in place of startuporders. But stratos is tightly coupled with > subscription to cluster as one to one mapping and also, load balancer uses > one to one mapping between cluster and hostname. So, if we are to bring up > new clusters/groups, then things might get complicated in stratos. As i > explained in the [part-1] discussion, we thought of achieving scale by > group member and scale by group using constructing the deployment policy in > a more advanced manner. I will start a separate thread on that. According > to all of our opinion, we can decide on how to follow that up. > > IMHO the startupOrders in composite application and group definitions > (json ) should look like > > "startupOrders": [ > “postgresGroup, php", > "sqlGroup, tomcat", > "tomcat, apimanager", > "tomcat, esb” > ] > > while the runtime representation of the logical relationship model for > each group or cartridge should use the corresponding aliases (or instance > Id) so the monitor will reference the aliases (or instance Ids) while the > json application / group definition will reference the group name (or type) > and cartridge type to define the dependencies, WDYT ? > > 2. If for example a cartridge has multiple dependencies we would > just add another line to the the startupOrders : > > e.g. postgresGroup depends on php and abc would be > represented by: > "startupOrders": [ > "postgresGroup, php", > "postgresGroup, “abc” > …. > ] > > > > > > Otherwise I think the proposal looks good, +1 > > > > Thanks, > > Reka > > > > Thanks > > > > Martin > > > > *From:* Reka Thirunavukkarasu [mailto:r...@wso2.com] > *Sent:* Wednesday, October 01, 2014 5:58 AM > *To:* dev > *Cc:* Lakmal Warusawithana; Isuru Haththotuwa; Martin Eppel (meppel); > Udara Liyanage > *Subject:* [Grouping][Part-2] Composite Application Dependency Tree > building based in Autoscaler > > > > Hi > > > > As you aware, in the composite application we can define the depencies > between groups/cartridges. Autoscaler's responsible is to parse this > dependencies and build up a logical relationship model in order to handle > the dependency information among the child nodes. As we have the > hierarchical monitors in autoscaler, i propose to have dependencies > information in each monitor that they aware of (the immediate child only). > In that monitor, we need to identify the group/cartridge which can be > started in parallel. So that a monitor can look at it's dependency and > control it's immediate children based on that. Once all the children are > active, it can pass the control to it's parent. For Eg: > > > > If we take the top level in Composite application which has mysqlGroup, > postgresGroup, php, tomcat, apimanager and esb. If they have an alias > saying my + cartridge/groupName then we can define the dependency > information as follows: > > - myPhp depends on myPostgresGroup (means postgresGroup should > be started before php) > > - myTomcat depends on myMysqlGroup > > - myApiManager depends on myTomcat > > - myEsb depends on myTomcat > > > > Like wise, groups will define their own dependency as well. > > > > In that way, we need to represent these dependency information as part of > Composite Application definition/GroupDefinition. In order to represent > this dependency information given above for Composite Application, i would > suggest to have the following in Composite Application definition. > > > > "startupOrders": [ > > "mypostgresGroup, myphp", > > "mysqlGroup, mytomcat", > > "mytomcat, myapimanager", > > "mytomcat, myesb" > > ] > > > > You can use the same format in GroupDefinition to define dependencies in a > group. > > > > As per the example, autoscaler will build a dependency tree for > ApplicationMonitor as below in order to identify the parallel and dependent > ones. So that Autoscaler will start up same level children monitors as in > parallel. > > > > > > > As above, ApplicationMonitor will start GroupMonitors of myPostgresGroup > and myMysqlGroup in parallel. Once postgres becomes active, > ApplicationMonitor will start ClusterMonitor for myPhp. Once > myPostgresGroup becomes active, ApplicationMonitor will start the immediate > child myTomcat. Once myTomcat becomes active, ApplicationMonitor will start > the myAppServer and myEsb in parallel. This will be applicable for > GroupMonitors as well. They can look at their own dependency tree and will > start their children. > > > > Please share your suggestions on the above model to handle the Dependency > information of Composite Application in autoscaler. > > > > > > Thanks, > > Reka > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > Reka Thirunavukkarasu > Senior Software Engineer, > WSO2, Inc.:http://wso2.com, > > Mobile: +94776442007 > > > > > > > > > > -- > > Reka Thirunavukkarasu > Senior Software Engineer, > WSO2, Inc.:http://wso2.com, > > Mobile: +94776442007 > > > > > > > > -- > > Reka Thirunavukkarasu > Senior Software Engineer, > WSO2, Inc.:http://wso2.com, > > Mobile: +94776442007 > > > > > > > > > -- > > Imesh Gunaratne > > > > Technical Lead, WSO2 > > Committer & PMC Member, Apache Stratos > > > > > > -- > > Reka Thirunavukkarasu > Senior Software Engineer, > WSO2, Inc.:http://wso2.com, > > Mobile: +94776442007 > > > > > > > > -- > > Reka Thirunavukkarasu > Senior Software Engineer, > WSO2, Inc.:http://wso2.com, > > Mobile: +94776442007 > > > -- Reka Thirunavukkarasu Senior Software Engineer, WSO2, Inc.:http://wso2.com, Mobile: +94776442007