Hi Lakmal, Application unemployment fails if application is not in DEPLOYED state. See the below error message in the logs. Yes as you mentioned we should Error/Invalid status should send as response if application in DEPLOYED (non removable) state.
*Could not delete application: [application-id] cisco-sample-vm* *org.apache.stratos.autoscaler.exception.AutoScalerException: Application is in deployed state, please undeploy it before deleting: [application-id] cisco-sample-vm* On Wed, Apr 8, 2015 at 3:50 PM, Lakmal Warusawithana <lak...@wso2.com> wrote: > Hi Udara > > Are we accepting remove call while application in un deployment process? > If does that wrong. We should return error in remove API call if > application not in created state IMO > > > On Wednesday, April 8, 2015, Udara Liyanage <ud...@wso2.com> wrote: > >> Hi Vanson, >> >> When I had a look at the logs attached. It seems that application removal >> is attempted immediately after the application undeployment(forceful). As I >> have already mentioned, even forceful undeployment takes a bit time since >> it invokes the graceful undeployment path. Inter component communication is >> done via event passing. Additionally forceful undeployment try once to >> terminate each instance. So you have to wait till undeployment finishes in >> order to remove the application. >> >> Yes, undeployment API call return just after it is invoked. But that does >> not implies undeployment is completed, it implies the undeployment job is >> accepted and will complete in future. You have to make another API call to >> get the status of the application and wait till it becomes CREATED to >> remove it. >> >> TID: [0] [STRATOS] [2015-04-08 02:19:57,979] INFO >> {org.apache.stratos.autoscaler.services.impl.AutoscalerServiceImpl} - >> Forcefully undeploying the application cisco-sample-vm >> TID: [0] [STRATOS] [2015-04-08 02:19:58,727] ERROR >> {org.apache.stratos.autoscaler.services.impl.AutoscalerServiceImpl} - >> Could not delete application: [application-id] cisco-sample-vm >> >> >> >> vm.cisco-sample-vm.domain8a869b76-0a7d-46af-8530-2ee4d23e5e3c >> TID: [0] [STRATOS] [2015-04-08 02:19:58,204] INFO >> {org.apache.stratos.cloud.controller.iaases.JcloudsIaas} - Starting to >> terminate member: [cartridge-type] cisco-sample-vm [member-id] >> cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain8a869b76-0a7d-46af-8530-2ee4d23e5e3c >> TID: [0] [STRATOS] [2015-04-08 02:19:58,727] ERROR >> {org.apache.stratos.autoscaler.services.impl.AutoscalerServiceImpl} - >> Could not delete application: [application-id] cisco-sample-vm. >> >> There is another exception in your log. I will have a loot at it and >> provide you an update. >> >> TID: [0] [STRATOS] [2015-04-08 02:20:45,074] ERROR >> {org.apache.stratos.cloud.controller.services.impl.InstanceCreator} - >> Could not start instance: [cartridge-type] cisco-sample-vm [cluster-id] >> cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain >> java.lang.NullPointerException >> at >> org.apache.stratos.cloud.controller.services.impl.InstanceCreator.run(InstanceCreator.java:78) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >> at java.lang.Thread.run(Thread.java:745) >> >> On Wed, Apr 8, 2015 at 8:09 AM, Vanson Lim <v...@cisco.com> wrote: >> >> On 4/7/15, 1:04 AM, Udara Liyanage wrote: >> >> Hi Vanson, >> >> After analyzing the logs you have sent we could find the issue. >> >> 1) AS acquire logs for the cluster, after that >> >> *AS sends ClusterstatusTerminatedEvent >> *CC captures the above event and publish ClusterTerminatedEvent >> *AS act upon the ClusterTerminatedEvent event and remove the monitor and >> clusters. >> When clusters are removed, all the locks related to the cluster is also >> removed. >> >> If all above steps occurred before AS release the lock, the above >> mentioned error occurs since the lock is removed. >> >> Since the events are asynchronous, there is a chance that it might >> occur. We could reproduce it after adding a sleep before releasing the >> lock. We made a fix 8b4b615e96e855672d00cfa205b73725894385e6. Please take >> an update and see if the issue is appearing again. >> >> >> Udara, >> >> Still not having any luck with this. The rest API's are returning >> Success, but the wso2carbon.log file shows some errors. My test is running >> in a tight loops issuing a application: >> add/deploy/undeploy(force)/remove, Each time through the loop the >> instances is not being properly terminated so I am ending up with a lot of >> orphaned instances that stratos doesn't know about. >> >> wso2carbon.log attached. >> >> -Vanson >> >> >> >> >> >> On Tue, Apr 7, 2015 at 9:34 AM, Isuru Haththotuwa <isu...@apache.org> >> wrote: >> >> Hi Vanson, >> >> On Mon, Apr 6, 2015 at 11:38 PM, Vanson Lim <v...@cisco.com> wrote: >> >> On 4/4/15, 12:13 PM, Udara Liyanage wrote: >> >> Hi, >> >> I added forcefull undeployment which can be invoked as below. >> curl -X POST -H "Content-Type: application/json" -k -v -u admin:admin >> https://$ >> {host_ip}:${host_port}/api/applications/${application_name}/undeploy? >> *force=true* >> >> >> Udara, >> >> I am not having any luck with the forceful delete api. I repeated try to >> run the following 4 steps: >> >> 1) add application >> 2) deploy application >> 3) undeploy application (force=true) >> 4) remove application >> >> Running steps 3 and 4 immediately after steps 1 & 2 seems to put stratos >> into a weird state, after which I can't can't reliably delete or add. >> Attached is the wso2carbon.log containing >> the traceback when I try steps 3&4 and see an error. >> >> This is a threading issue. I will also look in to this. >> >> >> -Vanson >> >> This is supposed to be executed when graceful undeployment does not >> complete successfully. This will try to terminate the instances once and >> send MemberTerminatedEvent regardless of termination process is successful >> or not. At the end Stratos should cleanup its internal memory and >> application should become CREATED. If stratos could not terminate instance, >> they should be manually terminated. >> >> Forceful termination can be invoked even without invoking graceful >> undeployment. >> >> >> On Tue, Mar 31, 2015 at 9:15 AM, Lakmal Warusawithana <lak...@wso2.com> >> wrote: >> >> >> >> On Mon, Mar 30, 2015 at 7:18 PM, Vanson Lim <v...@cisco.com> wrote: >> >> On 3/30/15, 9:18 AM, Lakmal Warusawithana wrote: >> >> IMO forceful un deploy is enough no need to have forceful delete. If un >> deployment success delete should work without any issues >> >> >> I agree, forceful undeploy should be enough. I assume this means that >> if I call undeploy (force=false) the first time and it doesn't work, I can >> call undeploy(force=true). >> >> In order for these api's to be robust, the create/deploy APIs should >> perform strict error checking and clean up of partial state when an >> exceptions are thrown. The code seems to do sufficient error checking but >> we keep encountering partial state left over when an exception is thrown. >> >> The undeploy force and/or delete APIs should ideally always cleanup even >> when detecting inconsistencies. We seem to be voting to handle this as a >> part of undeploy (force=true). I hope this means that an app MUST be >> cleanly transitioned back to "Created" state when we use (undeploy=force) >> and gracefully handle a subsequent deploy command. >> >> >> Yes, this is exactly what we need. >> >> >> -Vanson >> >> >> >> >> >> On Monday, March 30, 2015, Imesh Gunaratne <im...@apache.org> wrote: >> >> Thanks Vanson for pointing this scenario! Yes we would need to forcefully >> delete an application in certain scenarios. >> >> Udara: +1 For the suggestion! >> >> On Monday, March 30, 2015, Udara Liyanage <ud...@wso2.com> wrote: >> >> Hi, >> >> Sorry for the typo. We can add the flag as a query parameter. >> >> curl -X POST applications/single-cartridge-app/undeploy?*force=true* >> >> On Mon, Mar 30, 2015 at 10:04 AM, Udara Liyanage <ud...@wso2.com> wrote: >> >> Hi Imesh, >> >> How about the blow addition to the API? >> >> curl -X POST applications/single-cartridge-app/undeploy/*force=true* >> >> On Mon, Mar 30, 2015 at 9:48 AM, Udara Liyanage <ud...@wso2.com> wrote: >> >> >> >> On Sun, Mar 29, 2015 at 10:34 PM, Vanson Lim <v...@cisco.com> wrote: >> >> On 3/27/15, 5:57 PM, Vanson Lim wrote: >> >> On 3/27/15, 5:52 PM, Imesh Gunaratne wrote: >> >> Hi Vanson, >> >> I added a validation in Autoscaler to avoid applications being deleted >> while they are being undeployed in the background. >> >> Now the following error would be raised if this condition is met: >> "Application cannot be deleted, undeployment process is still in progress: >> [application-id] <application-id>". >> >> In addition I have implemented your suggestion to set the application >> status to CREATED once the application is completely undeployed. Now we can >> check from the API whether the application is undeployed properly. Please >> take a pull and verify. >> >> >> >> >> Took me a while to experiment with the failure cases. >> >> Some observations: >> >> 1) The change to check that the Application state changes to CREATED >> works. I am able to check that application status returns to Created before >> deleting. >> >> 2) Application delete throws tracebacks to the wso2carbon.log. IMO, >> this should only be informational rather then throw exceptions. >> >> I guess the reason for throwing an RuntimeException is to notify SM that >> an problem has occured while deleting the application. I will investigate >> firther. >> >> >> ie >> >> org.apache.stratos.autoscaler.exception.AutoScalerException: Application >> is in deployed state, please undeploy it before deleting: [application-id] >> cisco-sample-vm >> at >> org.apache.stratos.autoscaler.services.impl.AutoscalerServiceImpl.deleteApplication(AutoscalerServiceImpl.java:411) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:606) >> at >> org.apache.axis2.rpc.receivers.RPCUtil.invokeServiceClass(RPCUtil.java:212) >> at >> org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver.invokeBusinessLogic(RPCInOnlyMessageReceiver.java:66) >> at >> org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:110) >> at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180) >> at >> org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:172) >> at >> org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:146) >> at >> org.wso2.carbon.core.transports.CarbonServlet.doPost(CarbonServlet.java:231) >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:755) >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:848) >> at >> org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61) >> at >> org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128) >> at >> org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68) >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:848) >> at >> org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68) >> at >> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305) >> at >> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) >> at >> org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:61) >> at >> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243) >> at >> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) >> at >> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222) >> at >> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123) >> at >> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472) >> at >> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171) >> at >> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99) >> at >> org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:178) >> at >> org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47) >> at >> org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:56) >> at >> org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47) >> at >> org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:141) >> at >> org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:156) >> at >> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936) >> at >> org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:52) >> at >> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118) >> at >> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407) >> at >> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004) >> at >> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589) >> at >> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >> at java.lang.Thread.run(Thread.java:745) >> TID: [0] [STRATOS] [2015-03-28 00:17:52,952] ERROR >> {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver} - Could not >> delete application: [application-id] cisco-sample-vm >> java.lang.RuntimeException: Could not delete application: >> [application-id] cisco-sample-vm >> at >> org.apache.stratos.autoscaler.services.impl.AutoscalerServiceImpl.deleteApplication(AutoscalerServiceImpl.java:426) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:606) >> at >> org.apache.axis2.rpc.receivers.RPCUtil.invokeServiceClass(RPCUtil.java:212) >> at >> org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver.invokeBusinessLogic(RPCInOnlyMessageReceiver.java:66) >> at >> org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:110) >> at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180) >> at >> org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:172) >> at >> org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:146) >> at >> org.wso2.carbon.core.transports.CarbonServlet.doPost(CarbonServlet.java:231) >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:755) >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:848) >> at >> org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61) >> at >> org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128) >> at >> org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68) >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:848) >> at >> org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68) >> at >> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305) >> at >> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) >> at >> org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:61) >> at >> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243) >> at >> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) >> at >> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222) >> at >> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123) >> at >> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472) >> at >> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171) >> at >> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99) >> at >> org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:178) >> at >> org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47) >> at >> org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:56) >> at >> org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47) >> at >> org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:141) >> at >> org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:156) >> at >> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936) >> at >> org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:52) >> at >> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118) >> at >> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407) >> at >> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004) >> at >> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589) >> at >> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >> at java.lang.Thread.run(Thread.java:745) >> Caused by: org.apache.stratos.autoscaler.exception.AutoScalerException: >> Application is in deployed state, please undeploy it before deleting: >> [application-id] cisco-sample-vm >> at >> org.apache.stratos.autoscaler.services.impl.AutoscalerServiceImpl.deleteApplication(AutoscalerServiceImpl.java:411) >> ... 44 more >> >> >> 3) I am hitting the "redeploy application issue" that was fixed yesterday >> as a part of testing this fix and found that the application is in a >> broken state after an application redeploy such that application delete >> doesn't work anymore because the partially deployed application is in stuck >> in the "Deployed" state. >> >> This is similar to a valid use case which application delete doesn't >> currently handle. Our current implementation of application delete only >> works gracefully if the application is well behaved. If we have a single >> VM that is unresponsive to a terminate event, won't the application >> permanently be stuck in a "Deployed" state? This leads me to think that >> application delete needs to able to clean up all the state in the various >> components that make up stratos instead of preventing the delete requests >> regardless of the cluster instance counts or application status. In other >> words, the request should always be honored, otherwise we have no way of >> forcing the undeploy and deletion of an application. If the user wants to >> be graceful, they have the option to poll the application state changes >> using the application GET api before calling the application delete API. >> >> >> Your point seems to be valid since with the above fix, application >> will not be able to be deleted if it doesn't become created for some >> reason. One solution is to let the user explicitly specify to delete >> forcefully which will delete the application regardless of the state. I >> will do a further analysis and get back to you. >> >> >> -Vanson >> >> okay, will give it a try. >> >> -Vanson >> >> Thanks >> >> On Sat, Mar 28, 2015 at 2:40 AM, Vanson Lim <v...@cisco.com> wrote: >> >> On 3/27/15, 2:26 PM, Imesh Gunaratne wrote: >> >> Hi Vanson, >> >> I analyzed this issue. Logically we cannot delete an application soon >> after un-deploying it. The reason is that the un-deployment process >> executes gracefully on each member in all the clusters in an application. >> Depending on the complexity of the application, it may take some time to >> completely terminate all the members, remove the clusters and then >> un-deploy the application. >> >> IMO what we are missing here is a validation in the delete application >> API method, to tell the API client that an application cannot be deleted >> until it is completely un-deployed. >> >> Imesh, >> >> Thanks for looking at this. >> >> It sounds reasonable to hold off the removing to application and also >> have the remove rest api call return that it's not ready. For >> applications consisting of a lot of groups and cartridges the time to clean >> up could be variable. Could we use the Rest api to get the application >> status, ie check that status=Created before deleting. >> >> ie this api: >> >> >> https://cwiki.apache.org/confluence/display/STRATOS/4.1.0+Getting+Details+of+an+Application+via+REST+API >> >> It looks like an application has two states: "Created" and >> "Deployed". If we had a Undeploying state, then we could check and hold >> off >> requesting the remove-application until it returns to the Created state. >> >> -Vanson >> >> >> Thanks >> Imesh >> >> On Wed, Mar 25, 2015 at 12:10 PM, Udara Liyanage <ud...@wso2.com> wrote: >> >> Hi, >> >> This arises when the application is deleted immediately after the >> application underemployment. I guess CC does not have the member context >> when the application is removed. I will have a look. >> >> A simple workaround it to wait sometime to remove application after >> undeployment. >> >> On Tue, Mar 24, 2015 at 9:06 AM, Imesh Gunaratne <im...@apache.org> >> wrote: >> >> Thanks Martin! We will have a look at this. >> >> On Tue, Mar 24, 2015 at 1:21 AM, Martin Eppel (meppel) <mep...@cisco.com> >> wrote: >> >> I opened a jira https://issues.apache.org/jira/browse/STRATOS-1281 to >> track the issue, >> >> >> >> Thanks >> >> >> >> Martin >> >> >> >> *From:* Imesh Gunaratne [mailto:im...@apache.org] >> *Sent:* Friday, March 20, 2015 3:27 AM >> *To:* dev >> *Subject:* Re: Stratos 4.1.0 - tracebacks seen when issuing application >> undeploy/remove >> >> >> >> Hi Vanson, >> >> >> >> Thanks for reporting this problem. According to the logs stratos is >> trying to remove the same member twice in this flow: >> >> >> >> TID: [0] [STRATOS] [2015-03-19 19:45:43,952] INFO >> {org.apache.stratos.messaging.message.processor.application.ApplicationDeletedMessageProcessor} >> - [Application] cisco-sample-vm has been successfully removed >> >> TID: [0] [STRATOS] [2015-03-19 19:45:44,786] INFO >> {org.apache.stratos.cloud.controller.iaases.JcloudsIaas} - Member >> terminated: [member-id] >> cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain41624077-30c4-4bfe-a2a8-ece2fc4f550d >> >> >> >> TID: [0] [STRATOS] [2015-03-19 19:45:50,012] INFO >> {org.apache.stratos.common.client.CloudControllerServiceClient} - >> Terminating instance via cloud controller: [member] >> cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain41624077-30c4-4bfe-a2a8-ece2fc4f550d >> >> TID: [0] [STRATOS] [2015-03-19 19:45:50,017] ERROR >> {org.apache.stratos.cloud.controller.services.impl.CloudControllerServiceImpl} >> - Could not terminate instance, member context not found: [member-id] >> cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain41624077-30c4-4bfe-a2a8-ece2fc4f550d >> >> >> >> We might need to investigate and see what's causing this. >> >> >> >> Thanks >> >> >> >> >> >> On Fri, Mar 20, 2015 at 6:55 AM, Vanson Lim <v...@cisco.com> wrote: >> >> Hi, >> >> We are testing the behavior of stratos 4.1.0 rest api's and found that >> if we issue an undeploy-application followed immediately with >> application-remove, that we see the following traceback. >> >> The VM successfully get's deleted but not sure what kind of side effects >> this has on the system. >> >> Here's the snippet from the wso2carbon.log file. I've also attached the >> entire log. >> >> Steps to reproduce this: >> >> 1) deploy an application which leads to startup a single instance of a >> cartridge. >> 2) wait for it to become active >> 3) issue the undeploy application and application remove rest api calls. >> >> >> -Vanson >> >> >> >> TID: [0] [STRATOS] [2015-03-19 19:45:08,681] INFO >> {org.wso2.carbon.databridge.core.DataBridge} - admin connected >> TID: [0] [STRATOS] [2015-03-19 19:45:43,023] INFO >> {org.apache.stratos.autoscaler.services.impl.AutoscalerServiceImpl} - >> Starting to undeploy application: [application-id] cisco-sample-vm >> TID: [0] [STRATOS] [2015-03-19 19:45:43,024] INFO >> {org.apache.stratos.autoscaler.services.impl.AutoscalerServiceImpl} - >> Removing application signup: [application-id] cisco-sample-vm >> TID: [0] [STRATOS] [2015-03-19 19:45:43,051] INFO >> {org.apache.stratos.manager.components.ApplicationSignUpHandler} - Removing >> application signup: [application-id] cisco-sample-vm [tenant-id] -1234 >> TID: [0] [STRATOS] [2015-03-19 19:45:43,084] INFO >> {org.apache.stratos.manager.components.ApplicationSignUpHandler} - >> Application signup removed successfully: [application-id] cisco-sample-vm >> [tenant-id] -1234 >> TID: [0] [STRATOS] [2015-03-19 19:45:43,091] INFO >> {org.apache.stratos.autoscaler.context.AutoscalerContext} - Network >> partition algorithm context is removed successfully: [id] cisco-sample-vm >> TID: [0] [STRATOS] [2015-03-19 19:45:43,092] INFO >> {org.apache.stratos.autoscaler.monitor.cluster.ClusterMonitor} - Publishing >> Cluster terminating event for [application] cisco-sample-vm [cluster] >> cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain [instance] >> cisco-sample-vm-1 >> TID: [0] [STRATOS] [2015-03-19 19:45:43,109] INFO >> {org.apache.stratos.cloud.controller.messaging.topology.TopologyBuilder} - >> Cluster Terminating adding status started >> forcisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain >> TID: [0] [STRATOS] [2015-03-19 19:45:43,117] INFO >> {org.apache.stratos.autoscaler.applications.topic.ApplicationsEventPublisher} >> - Publishing application inactivated event: [application] cisco-sample-vm >> [instance] cisco-sample-vm-1 >> TID: [0] [STRATOS] [2015-03-19 19:45:43,120] INFO >> {org.apache.stratos.cloud.controller.messaging.publisher.TopologyEventPublisher} >> - Publishing Cluster terminating event: [application-id] cisco-sample-vm >> [cluster id] cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain >> [instance-id] cisco-sample-vm-1 >> TID: [0] [STRATOS] [2015-03-19 19:45:43,144] INFO >> {org.apache.stratos.autoscaler.services.impl.AutoscalerServiceImpl} - >> Application undeployed successfully: [application-id] cisco-sample-vm >> TID: [0] [STRATOS] [2015-03-19 19:45:43,154] INFO >> {org.apache.stratos.autoscaler.event.receiver.topology.AutoscalerTopologyEventReceiver} >> - [ClusterTerminatingEvent] Received: class >> org.apache.stratos.messaging.event.topology.ClusterInstanceTerminatingEvent >> TID: [0] [STRATOS] [2015-03-19 19:45:43,155] INFO >> {org.apache.stratos.autoscaler.event.publisher.InstanceNotificationPublisher} >> - Publishing Instance Cleanup Event: [cluster] >> cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain >> TID: [0] [STRATOS] [2015-03-19 19:45:43,171] WARN >> {org.apache.stratos.autoscaler.status.processor.cluster.ClusterStatusActiveProcessor} >> - No possible state change found for [type] [cluster] >> cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain [instance] >> TID: [0] [STRATOS] [2015-03-19 19:45:43,192] INFO >> {org.apache.stratos.cloud.controller.messaging.topology.TopologyBuilder} >> >> > > -- > Sent from Gmail Mobile > -- Udara Liyanage Software Engineer WSO2, Inc.: http://wso2.com lean. enterprise. middleware web: http://udaraliyanage.wordpress.com phone: +94 71 443 6897