On Wed, Feb 22, 2012 at 10:16 AM, Nirmal Fernando <[email protected]> wrote: > > > On Wed, Feb 22, 2012 at 9:50 AM, Selvaratnam Uthaiyashankar > <[email protected]> wrote: >> >> On Wed, Feb 22, 2012 at 8:27 AM, Nirmal Fernando <[email protected]> wrote: >> > Hi Shankar, >> > >> > On Wed, Feb 22, 2012 at 8:12 AM, Selvaratnam Uthaiyashankar >> > <[email protected]> wrote: >> >> >> >> Hi Nirmal, >> >> >> >> On Tue, Feb 21, 2012 at 11:04 AM, Nirmal Fernando <[email protected]> >> >> wrote: >> >> > >> >> > >> >> > On Tue, Feb 21, 2012 at 10:18 AM, Nirmal Fernando <[email protected]> >> >> > wrote: >> >> >> >> >> >> Hi All, >> >> >> >> >> >> I need your views on following. >> >> >> >> >> >> In the Agent side we need to have a configuration file (say >> >> >> instances_config.xml) which specifies the paths and names of the >> >> >> instances >> >> >> belong to domains. >> >> >> >> >> >> I thought to have following structure: >> >> >> >> >> > Value of the path attribute should change like this: >> >> > >> >> >> >> >> >> <domain name="wso2.as.domain"> >> >> >> <instance name="wso2as-4.1.0-SNAPSHOT" >> >> >> >> >> >> path="/home/nirmal/Desktop/wso2as-4.1.0-SNAPSHOT/bin/wso2server.sh"/> >> >> >> <instance name="wso2as-4.1.0-SNAPSHOT" >> >> >> path="/home/nirmal/Temp/wso2as-4.1.0-SNAPSHOT/bin/wso2server.sh"/> >> >> >> >> >> >> ..... >> >> >> </domain> >> >> >> <domain name="wso2.bps.domain"> >> >> >> <instance name="wso2bps-4.1.0-SNAPSHOT" >> >> >> >> >> >> path="/home/nirmal/Desktop/wso2bps-4.1.0-SNAPSHOT/bin/wso2server.sh"/> >> >> >> <instance name="wso2bps-4.1.0-SNAPSHOT" >> >> >> path="/home/nirmal/Temp/wso2bps-4.1.0-SNAPSHOT/bin/wso2server.sh"/> >> >> >> >> >> >> ..... >> >> >> </domain> >> >> >> >> >> >> What are these two instances? Why they should be in the configuration >> >> file? If they are the paths of instances that can be started, then if >> >> I want to start a third instance what should I do? >> > >> > >> > Agents should know the instances that it can spawn. Thus, each Agent >> > machine >> > has a configuration file which points to those instances. An Agent can >> > only >> > spawn instances it has. >> > >> > This configuration file will be loaded each time an Agent get >> > registered. So >> > if you want to add a new instance, you can unregister the Agent, edit >> > the >> > configuration file and re-register. Then the Agent Service will locate >> > the >> > newly added instance and list it as an instance that can be spawned. >> >> This might become a maintanance problem. Suppose I have a machine >> which can run 5 instance of stratos service. This service can be ESB, >> AppServer, GS, MS, Manager, .... We have 13 service, so I have to give >> 13 * 5 configuration. > > > Understood the point! > >> >> >> I would like to give the location of binary pack location for 13 >> services, and when an instance is required, agent can copy the pack >> into its "working directory" and run it. So, actual instance location >> is not needed and will be handled by agent. This is the normal way >> most of the software work (e.g. eucalyptus). In this method, the >> configuration for all JVM autoscale agent will be same. (can copy the >> binary pack in same location, working directory can be same location). >> Based on the request, Agent will copy and start the "instace" >> dynamically. >> >> WDYT? > > > Sounds cool! I have some suggestions though. > > Since there should be a map from domain name to instance, is it ok to > specify domain name > alone with the path? > > So in configuration file we have: > ..... > ..... > <instance domain="wso2.as.domain" path="/home/a/wso2as-4.1.0-SNAPSHOT"/> > <instance domain="wso2.bps.domain" path="/home/a/wso2bps-4.1.0-SNAPSHOT"/> > .... > .... > > and when Agent copies an instance from this location to it's working > directory, it should rename it by adding a suffix to "wso2xx-4.1.0-SNAPSHOT" > eg: wso2as-4.1.0-SNAPSHOT-1, otherwise we want be able to spawn more than > one instance.
+1! Shankar > > > >> >> >> Shankar >> >> >> > >> > WDYT? >> > >> >> >> >> >> >> Shankar >> >> >> >> >> >> >> ..... >> >> >> >> >> >> WDYT? >> >> >> >> >> >> PS: It is my understanding that the autoscaler component passes the >> >> >> 'domain name' to the "Autoscaler Service", when it wants to scale. >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> On Mon, Feb 20, 2012 at 9:31 PM, Selvaratnam Uthaiyashankar >> >> >> <[email protected]> wrote: >> >> >>> >> >> >>> On Mon, Feb 20, 2012 at 8:17 PM, Nirmal Fernando <[email protected]> >> >> >>> wrote: >> >> >>> > Hi Shankar, >> >> >>> > >> >> >>> > On Mon, Feb 20, 2012 at 5:34 PM, Selvaratnam Uthaiyashankar >> >> >>> > <[email protected]> wrote: >> >> >>> >> >> >> >>> >> Hi Nirmal, >> >> >>> >> >> >> >>> >> Good progress!. we can do a review on the work already done. >> >> >>> > >> >> >>> > >> >> >>> > Thanks, will arrange a review this week. >> >> >>> > >> >> >>> >> >> >> >>> >> >> >> >>> >> Did you try the agent in windows? >> >> >>> > >> >> >>> > >> >> >>> > No, I didn't! >> >> >>> > >> >> >>> >> >> >> >>> >> Is it using any native code? >> >> >>> > >> >> >>> > >> >> >>> > It needs to use native code in cases like spawning a new JVM (in >> >> >>> > Ubuntu >> >> >>> > we >> >> >>> > need /bin/sh to run the script 'wso2server.sh' in windows we need >> >> >>> > to >> >> >>> > run the >> >> >>> > batch file. >> >> >>> > But I believe this can be handled within the code it self by >> >> >>> > checking >> >> >>> > the OS >> >> >>> > name and execute appropriate commands! >> >> >>> > >> >> >>> >> >> >> >>> >> >> >> >>> >> See some comments below. >> >> >>> >> >> >> >>> >> On Sat, Feb 18, 2012 at 11:10 PM, Nirmal Fernando >> >> >>> >> <[email protected]> >> >> >>> >> wrote: >> >> >>> >> > Hi All, >> >> >>> >> > >> >> >>> >> > I've started to implement 'new autoscaler architecture' [1] on >> >> >>> >> > February >> >> >>> >> > 8th. >> >> >>> >> > After having a discussion with Azeez, we came up with three >> >> >>> >> > milestones >> >> >>> >> > for >> >> >>> >> > the first iteration. >> >> >>> >> > >> >> >>> >> > >> >> >>> >> > >> >> >>> >> > >> >> >>> >> > ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- >> >> >>> >> > >> >> >>> >> > M1: Make a WS call spawns a new JVM instance & make the new >> >> >>> >> > instance >> >> >>> >> > joins >> >> >>> >> > the cluster with hard coded Agents. >> >> >>> >> > >> >> >>> >> > M2: Finish Agent registration related parts and spawn a new >> >> >>> >> > instance >> >> >>> >> > in >> >> >>> >> > a >> >> >>> >> > selected Agent. >> >> >>> >> > >> >> >>> >> > M3: Let current autoscaler component calls 'AutoscalerService' >> >> >>> >> > and >> >> >>> >> > spawns a >> >> >>> >> > new JVM instance in a selected Agent. Also applying WS >> >> >>> >> > security. >> >> >>> >> > >> >> >>> >> > >> >> >>> >> > >> >> >>> >> > >> >> >>> >> > >> >> >>> >> > ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- >> >> >>> >> > >> >> >>> >> > I've attached set of images which depicts different stages of >> >> >>> >> > new >> >> >>> >> > architecture. Reading order of images is as follows: >> >> >>> >> > >> >> >>> >> > agent-reg --> autoscaler --> spawn --> cluster >> >> >>> >> > >> >> >>> >> > >> >> >>> >> > M1 is already finished. >> >> >>> >> > >> >> >>> >> > I've almost done the Agent registration (M2) aspect. >> >> >>> >> > >> >> >>> >> > What it does now? >> >> >>> >> > >> >> >>> >> > AgentManagementService and AutoscalerService should be >> >> >>> >> > deployed >> >> >>> >> > in >> >> >>> >> > the >> >> >>> >> > back-end server. (say: IP is 192.168.1.2) >> >> >>> >> > >> >> >>> >> > AgentService should be deployed in machines (i.e. where new >> >> >>> >> > JVM >> >> >>> >> > instances >> >> >>> >> > will be spawned). (say: IP is 192.168.1.3) >> >> >>> >> > >> >> >>> >> > AgentService do a web service call (eg: curl -X GET >> >> >>> >> > http://localhost:8080/axis2/services/AgentService/agent) to >> >> >>> >> > AgentManagementService and get registered. Then >> >> >>> >> > AgentManagementService >> >> >>> >> > stores the URL (eg: >> >> >>> >> > http://192.168.1.3:8080/axis2/services/AgentService/) of >> >> >>> >> > that particular request. >> >> >>> >> > >> >> >>> >> > When autoscaler API (implementing this is M3) wanted to spawn >> >> >>> >> > a >> >> >>> >> > new >> >> >>> >> > instance >> >> >>> >> > (JVM or EC2 or anything else), it should call >> >> >>> >> > AutoscalerService >> >> >>> >> > (eg: >> >> >>> >> > curl >> >> >>> >> > --data "instanceName=as" -X POST >> >> >>> >> > >> >> >>> >> > >> >> >>> >> > http://192.168.1.2:8080/axis2/services/AutoscalerService/instance). >> >> >>> >> > Then >> >> >>> >> > AutoscalerService decides which kind of instance to spawn. >> >> >>> >> > >> >> >>> >> > If AutoscalerService decides to spawn a JVM instance, it >> >> >>> >> > creates >> >> >>> >> > a >> >> >>> >> > new >> >> >>> >> > JVMAdaptor instance. JVMAdaptor then calls (eg: curl -X GET >> >> >>> >> > >> >> >>> >> > >> >> >>> >> > http://192.168.1.2:8080/axis2/services/AgentManagementService/agent) >> >> >>> >> > AgentManagementService in order to pick an Agent. >> >> >>> >> > AgentManagementService >> >> >>> >> > picks an Agent in a round robin manner and sends back that >> >> >>> >> > Agent's >> >> >>> >> > EPR >> >> >>> >> > (http://192.168.1.3:8080/axis2/services/AgentService/) to >> >> >>> >> > JVMAdaptor. >> >> >>> >> > Then >> >> >>> >> > JVMAdaptor does a web service call to that particular >> >> >>> >> > AgentService >> >> >>> >> > and >> >> >>> >> > asked >> >> >>> >> > it to spawn the requested instance (instance is yet hard >> >> >>> >> > coded). >> >> >>> >> > >> >> >>> >> > Spawned instance then joins the cluster. >> >> >>> >> > >> >> >>> >> > >> >> >>> >> > Current observations: >> >> >>> >> > >> >> >>> >> > Currently starting instance aspect is done. >> >> >>> >> > >> >> >>> >> > I've tested this using 2 machines, and starting instance works >> >> >>> >> > fine. >> >> >>> >> > >> >> >>> >> > It is observed that the spawned JVM instances get killed when >> >> >>> >> > you >> >> >>> >> > issue >> >> >>> >> > 'ctrl+c' in the terminal where axis2 running. But when I >> >> >>> >> > killed >> >> >>> >> > axis2server.sh process only, the new JAVA process wasn't get >> >> >>> >> > killed. >> >> >>> >> > To >> >> >>> >> > avoid process get killing when issue ctrl+c, we might need to >> >> >>> >> > handle >> >> >>> >> > SIGINT >> >> >>> >> > [2]. Can't we be happy with the fact that it doesn't get >> >> >>> >> > killed >> >> >>> >> > after >> >> >>> >> > killing axis2server.sh? >> >> >>> >> >> >> >>> >> JVMs (which were created) should live even if you kill the >> >> >>> >> agent. >> >> >>> >> >> >> >>> > >> >> >>> > Yes, but the problem is how you kill! If you use kill -9 and >> >> >>> > kills >> >> >>> > Axis2 >> >> >>> > related process (Axis2 is where Agent Service is deployed), >> >> >>> > spawned >> >> >>> > JVM >> >> >>> > instances aren't get killed. >> >> >>> > Anyway will research a bit more, I understand that spawned >> >> >>> > instance >> >> >>> > should >> >> >>> > never get killed, unless it is asked to! >> >> >>> > >> >> >>> >> >> >> >>> >> > >> >> >>> >> > >> >> >>> >> > Work to be done: >> >> >>> >> > >> >> >>> >> > Terminating and finding instance aspects should be addressed. >> >> >>> >> >> >> >>> >> You have to keep track of which JVMs/VMs created by the >> >> >>> >> autoscaler. >> >> >>> >> Otherwise you will not be able to stop them when you want to >> >> >>> >> scale >> >> >>> >> down. I believe, this information should be stored/kept with >> >> >>> >> "autoscaler service". >> >> >>> > >> >> >>> > >> >> >>> > I have started to implement this part today! According to the >> >> >>> > design >> >> >>> > 'Autoscaler Service' is only responsible for >> >> >>> > deciding the type of instance it should be spawned. That is a JVM >> >> >>> > instance >> >> >>> > or an EC2 instance or something else. Thus, it keeps track of >> >> >>> > the type of instance spawned. >> >> >>> > >> >> >>> > It is JVM Adaptor which keeps track of instances spawned in each >> >> >>> > Agent. >> >> >>> >> >> >>> +1. It should be JVM adapter for JVM instances and EC2 adapter for >> >> >>> EC2 >> >> >>> instance, etc. >> >> >>> >> >> >>> Shankar >> >> >>> >> >> >>> >> >> >>> > >> >> >>> > And it is respective Agent Service that kills an instance, upon a >> >> >>> > request! >> >> >>> > >> >> >>> > >> >> >>> >> >> >> >>> >> >> >> >>> >> > The 'sad side' (when things go wrong) should be addressed. >> >> >>> >> > AgentServices should know EPR of AgentManagementService (this >> >> >>> >> > is >> >> >>> >> > hard >> >> >>> >> > coded >> >> >>> >> > now). Should read from a config file? >> >> >>> >> >> >> >>> >> +1 fore reading from config file. >> >> >>> > >> >> >>> > >> >> >>> > Great! >> >> >>> > >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> > JVMAdaptor should keep a map of Agent EPR to AgentService >> >> >>> >> > client, >> >> >>> >> > to >> >> >>> >> > avoid >> >> >>> >> > creating multiple clients to access the same Agent. >> >> >>> >> > AgentService should keep a count of spawned instances and get >> >> >>> >> > itself >> >> >>> >> > unregistered if it can't handle any new instances (threshold >> >> >>> >> > value >> >> >>> >> > should be >> >> >>> >> > decided). And when it feels (another threshold) that it can >> >> >>> >> > handle >> >> >>> >> > new >> >> >>> >> > instances it should register again. >> >> >>> >> > Autoscaler API should be designed. >> >> >>> >> > Should port EC2 client to EC2Adaptor. >> >> >>> >> > Should test the whole scenario. >> >> >>> >> > >> >> >>> >> > >> >> >>> >> > I appreciate your comments/thoughts on above facts. >> >> >>> >> > >> >> >>> >> > >> >> >>> >> > [1] >> >> >>> >> > >> >> >>> >> > >> >> >>> >> > >> >> >>> >> > http://mail.wso2.org/mailarchive/architecture/2011-October/006414.html >> >> >>> >> > >> >> >>> >> > [2] http://www.cons.org/cracauer/sigint.html >> >> >>> >> > >> >> >>> >> > >> >> >>> >> > >> >> >>> >> > -- >> >> >>> >> > >> >> >>> >> > Thanks & regards, >> >> >>> >> > Nirmal >> >> >>> >> > >> >> >>> >> > Software Engineer- Platform Technologies Team, WSO2 Inc. >> >> >>> >> > Mobile: +94715779733 >> >> >>> >> > Blog: http://nirmalfdo.blogspot.com/ >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> -- >> >> >>> >> S.Uthaiyashankar >> >> >>> >> Senior Architect & Senior Manager >> >> >>> >> WSO2 Inc. >> >> >>> >> http://wso2.com/ - "lean . enterprise . middleware" >> >> >>> >> >> >> >>> >> Phone: +94 714897591 >> >> >>> > >> >> >>> > >> >> >>> > >> >> >>> > >> >> >>> > -- >> >> >>> > >> >> >>> > Thanks & regards, >> >> >>> > Nirmal >> >> >>> > >> >> >>> > Software Engineer- Platform Technologies Team, WSO2 Inc. >> >> >>> > Mobile: +94715779733 >> >> >>> > Blog: http://nirmalfdo.blogspot.com/ >> >> >>> >> >> >>> >> >> >>> >> >> >>> -- >> >> >>> S.Uthaiyashankar >> >> >>> Senior Architect & Senior Manager >> >> >>> WSO2 Inc. >> >> >>> http://wso2.com/ - "lean . enterprise . middleware" >> >> >>> >> >> >>> Phone: +94 714897591 >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> -- >> >> >> >> >> >> Thanks & regards, >> >> >> Nirmal >> >> >> >> >> >> Software Engineer- Platform Technologies Team, WSO2 Inc. >> >> >> Mobile: +94715779733 >> >> >> Blog: http://nirmalfdo.blogspot.com/ >> >> > >> >> > >> >> > >> >> > >> >> > -- >> >> > >> >> > Thanks & regards, >> >> > Nirmal >> >> > >> >> > Software Engineer- Platform Technologies Team, WSO2 Inc. >> >> > Mobile: +94715779733 >> >> > Blog: http://nirmalfdo.blogspot.com/ >> >> >> >> >> >> >> >> -- >> >> S.Uthaiyashankar >> >> Senior Architect & Senior Manager >> >> WSO2 Inc. >> >> http://wso2.com/ - "lean . enterprise . middleware" >> >> >> >> Phone: +94 714897591 >> > >> > >> > >> > >> > -- >> > >> > Thanks & regards, >> > Nirmal >> > >> > Software Engineer- Platform Technologies Team, WSO2 Inc. >> > Mobile: +94715779733 >> > Blog: http://nirmalfdo.blogspot.com/ >> >> >> >> -- >> S.Uthaiyashankar >> Senior Architect & Senior Manager >> WSO2 Inc. >> http://wso2.com/ - "lean . enterprise . middleware" >> >> Phone: +94 714897591 > > > > > -- > > Thanks & regards, > Nirmal > > Software Engineer- Platform Technologies Team, WSO2 Inc. > Mobile: +94715779733 > Blog: http://nirmalfdo.blogspot.com/ -- S.Uthaiyashankar Senior Architect & Senior Manager WSO2 Inc. http://wso2.com/ - "lean . enterprise . middleware" Phone: +94 714897591 _______________________________________________ Carbon-dev mailing list [email protected] http://mail.wso2.org/cgi-bin/mailman/listinfo/carbon-dev
