Hi, This arises when the application is deleted immediately after the application underemployment. I guess CC does not have the member context when the application is removed. I will have a look.
A simple workaround it to wait sometime to remove application after undeployment. On Tue, Mar 24, 2015 at 9:06 AM, Imesh Gunaratne <im...@apache.org> wrote: > Thanks Martin! We will have a look at this. > > On Tue, Mar 24, 2015 at 1:21 AM, Martin Eppel (meppel) <mep...@cisco.com> > wrote: > >> I opened a jira https://issues.apache.org/jira/browse/STRATOS-1281 to >> track the issue, >> >> >> >> Thanks >> >> >> >> Martin >> >> >> >> *From:* Imesh Gunaratne [mailto:im...@apache.org] >> *Sent:* Friday, March 20, 2015 3:27 AM >> *To:* dev >> *Subject:* Re: Stratos 4.1.0 - tracebacks seen when issuing application >> undeploy/remove >> >> >> >> Hi Vanson, >> >> >> >> Thanks for reporting this problem. According to the logs stratos is >> trying to remove the same member twice in this flow: >> >> >> >> TID: [0] [STRATOS] [2015-03-19 19:45:43,952] INFO >> {org.apache.stratos.messaging.message.processor.application.ApplicationDeletedMessageProcessor} >> - [Application] cisco-sample-vm has been successfully removed >> >> TID: [0] [STRATOS] [2015-03-19 19:45:44,786] INFO >> {org.apache.stratos.cloud.controller.iaases.JcloudsIaas} - Member >> terminated: [member-id] >> cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain41624077-30c4-4bfe-a2a8-ece2fc4f550d >> >> >> >> TID: [0] [STRATOS] [2015-03-19 19:45:50,012] INFO >> {org.apache.stratos.common.client.CloudControllerServiceClient} - >> Terminating instance via cloud controller: [member] >> cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain41624077-30c4-4bfe-a2a8-ece2fc4f550d >> >> TID: [0] [STRATOS] [2015-03-19 19:45:50,017] ERROR >> {org.apache.stratos.cloud.controller.services.impl.CloudControllerServiceImpl} >> - Could not terminate instance, member context not found: [member-id] >> cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain41624077-30c4-4bfe-a2a8-ece2fc4f550d >> >> >> >> We might need to investigate and see what's causing this. >> >> >> >> Thanks >> >> >> >> >> >> On Fri, Mar 20, 2015 at 6:55 AM, Vanson Lim <v...@cisco.com> wrote: >> >> Hi, >> >> We are testing the behavior of stratos 4.1.0 rest api's and found that >> if we issue an undeploy-application followed immediately with >> application-remove, that we see the following traceback. >> >> The VM successfully get's deleted but not sure what kind of side effects >> this has on the system. >> >> Here's the snippet from the wso2carbon.log file. I've also attached the >> entire log. >> >> Steps to reproduce this: >> >> 1) deploy an application which leads to startup a single instance of a >> cartridge. >> 2) wait for it to become active >> 3) issue the undeploy application and application remove rest api calls. >> >> >> -Vanson >> >> >> >> TID: [0] [STRATOS] [2015-03-19 19:45:08,681] INFO >> {org.wso2.carbon.databridge.core.DataBridge} - admin connected >> TID: [0] [STRATOS] [2015-03-19 19:45:43,023] INFO >> {org.apache.stratos.autoscaler.services.impl.AutoscalerServiceImpl} - >> Starting to undeploy application: [application-id] cisco-sample-vm >> TID: [0] [STRATOS] [2015-03-19 19:45:43,024] INFO >> {org.apache.stratos.autoscaler.services.impl.AutoscalerServiceImpl} - >> Removing application signup: [application-id] cisco-sample-vm >> TID: [0] [STRATOS] [2015-03-19 19:45:43,051] INFO >> {org.apache.stratos.manager.components.ApplicationSignUpHandler} - Removing >> application signup: [application-id] cisco-sample-vm [tenant-id] -1234 >> TID: [0] [STRATOS] [2015-03-19 19:45:43,084] INFO >> {org.apache.stratos.manager.components.ApplicationSignUpHandler} - >> Application signup removed successfully: [application-id] cisco-sample-vm >> [tenant-id] -1234 >> TID: [0] [STRATOS] [2015-03-19 19:45:43,091] INFO >> {org.apache.stratos.autoscaler.context.AutoscalerContext} - Network >> partition algorithm context is removed successfully: [id] cisco-sample-vm >> TID: [0] [STRATOS] [2015-03-19 19:45:43,092] INFO >> {org.apache.stratos.autoscaler.monitor.cluster.ClusterMonitor} - Publishing >> Cluster terminating event for [application] cisco-sample-vm [cluster] >> cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain [instance] >> cisco-sample-vm-1 >> TID: [0] [STRATOS] [2015-03-19 19:45:43,109] INFO >> {org.apache.stratos.cloud.controller.messaging.topology.TopologyBuilder} - >> Cluster Terminating adding status started >> forcisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain >> TID: [0] [STRATOS] [2015-03-19 19:45:43,117] INFO >> {org.apache.stratos.autoscaler.applications.topic.ApplicationsEventPublisher} >> - Publishing application inactivated event: [application] cisco-sample-vm >> [instance] cisco-sample-vm-1 >> TID: [0] [STRATOS] [2015-03-19 19:45:43,120] INFO >> {org.apache.stratos.cloud.controller.messaging.publisher.TopologyEventPublisher} >> - Publishing Cluster terminating event: [application-id] cisco-sample-vm >> [cluster id] cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain >> [instance-id] cisco-sample-vm-1 >> TID: [0] [STRATOS] [2015-03-19 19:45:43,144] INFO >> {org.apache.stratos.autoscaler.services.impl.AutoscalerServiceImpl} - >> Application undeployed successfully: [application-id] cisco-sample-vm >> TID: [0] [STRATOS] [2015-03-19 19:45:43,154] INFO >> {org.apache.stratos.autoscaler.event.receiver.topology.AutoscalerTopologyEventReceiver} >> - [ClusterTerminatingEvent] Received: class >> org.apache.stratos.messaging.event.topology.ClusterInstanceTerminatingEvent >> TID: [0] [STRATOS] [2015-03-19 19:45:43,155] INFO >> {org.apache.stratos.autoscaler.event.publisher.InstanceNotificationPublisher} >> - Publishing Instance Cleanup Event: [cluster] >> cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain >> TID: [0] [STRATOS] [2015-03-19 19:45:43,171] WARN >> {org.apache.stratos.autoscaler.status.processor.cluster.ClusterStatusActiveProcessor} >> - No possible state change found for [type] [cluster] >> cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain [instance] >> TID: [0] [STRATOS] [2015-03-19 19:45:43,192] INFO >> {org.apache.stratos.cloud.controller.messaging.topology.TopologyBuilder} - >> member maintenance mode event adding status started >> TID: [0] [STRATOS] [2015-03-19 19:45:43,204] INFO >> {org.apache.stratos.cloud.controller.messaging.publisher.TopologyEventPublisher} >> - Publishing member in maintenance mode event: [service-name] >> cisco-sample-vm [cluster-id] >> cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain >> [cluster-instance-id] cisco-sample-vm-1 [member-id] >> cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain41624077-30c4-4bfe-a2a8-ece2fc4f550d >> [network-partition-id] RegionOne [partition-id] whole-region >> TID: [0] [STRATOS] [2015-03-19 19:45:43,230] INFO >> {org.apache.stratos.messaging.message.processor.topology.MemberMaintenanceModeProcessor} >> - Member updated as In_Maintenance: [service] cisco-sample-vm [cluster] >> cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain [member] >> cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain41624077-30c4-4bfe-a2a8-ece2fc4f550d >> TID: [0] [STRATOS] [2015-03-19 19:45:43,231] INFO >> {org.apache.stratos.cloud.controller.messaging.topology.TopologyBuilder} - >> Member Ready to shut down event adding status started >> TID: [0] [STRATOS] [2015-03-19 19:45:43,242] INFO >> {org.apache.stratos.cloud.controller.messaging.publisher.TopologyEventPublisher} >> - Publishing member ready to shut down event: [service-name] >> cisco-sample-vm [cluster-id] >> cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain >> [cluster-instance-id] cisco-sample-vm-1 [member-id] >> cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain41624077-30c4-4bfe-a2a8-ece2fc4f550d >> [network-partition-id] RegionOne [partition-id] whole-region >> TID: [0] [STRATOS] [2015-03-19 19:45:43,262] INFO >> {org.apache.stratos.messaging.message.processor.topology.MemberReadyToShutdownMessageProcessor} >> - Member updated as Ready to shutdown: [service] cisco-sample-vm [cluster] >> cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain [member] >> cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain41624077-30c4-4bfe-a2a8-ece2fc4f550d >> TID: [0] [STRATOS] [2015-03-19 19:45:43,264] INFO >> {org.apache.stratos.autoscaler.event.receiver.topology.AutoscalerTopologyEventReceiver} >> - [MemberReadyToShutdownEvent] Received: class >> org.apache.stratos.messaging.event.topology.MemberReadyToShutdownEvent >> TID: [0] [STRATOS] [2015-03-19 19:45:43,264] INFO >> {org.apache.stratos.autoscaler.context.partition.ClusterLevelPartitionContext} >> - Starting the moving of termination pending to obsolete for [member] >> cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain41624077-30c4-4bfe-a2a8-ece2fc4f550d >> TID: [0] [STRATOS] [2015-03-19 19:45:43,264] INFO >> {org.apache.stratos.autoscaler.context.partition.ClusterLevelPartitionContext} >> - Found termination pending member and trying to move [member] >> cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain41624077-30c4-4bfe-a2a8-ece2fc4f550d >> to obsolete list >> TID: [0] [STRATOS] [2015-03-19 19:45:43,266] INFO >> {org.apache.stratos.common.client.CloudControllerServiceClient} - >> Terminating instance via cloud controller: [member] >> cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain41624077-30c4-4bfe-a2a8-ece2fc4f550d >> TID: [0] [STRATOS] [2015-03-19 19:45:43,275] INFO >> {org.apache.stratos.cloud.controller.iaases.JcloudsIaas} - Starting to >> terminate member: [cartridge-type] cisco-sample-vm [member-id] >> cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain41624077-30c4-4bfe-a2a8-ece2fc4f550d >> TID: [0] [STRATOS] [2015-03-19 19:45:43,883] INFO >> {org.apache.stratos.messaging.domain.application.locking.ApplicationLockHierarchy} >> - Removed lock for application: [application-id] cisco-sample-vm >> TID: [0] [STRATOS] [2015-03-19 19:45:43,909] INFO >> {org.apache.stratos.cloud.controller.messaging.receiver.application.ApplicationEventReceiver} >> - Application deleted event received: [application-id] cisco-sample-vm >> TID: [0] [STRATOS] [2015-03-19 19:45:43,909] INFO >> {org.apache.stratos.cloud.controller.messaging.topology.TopologyBuilder} - >> Removed application [ cisco-sample-vm ]'s Cluster [ >> cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain ] from the topology >> TID: [0] [STRATOS] [2015-03-19 19:45:43,910] INFO >> {org.apache.stratos.autoscaler.services.impl.AutoscalerServiceImpl} - >> Application deleted successfully: [application-id] >> TID: [0] [STRATOS] [2015-03-19 19:45:43,930] INFO >> {org.apache.stratos.cloud.controller.messaging.publisher.TopologyEventPublisher} >> - Publishing application clusters removed event: [application-id] >> cisco-sample-vm >> TID: [0] [STRATOS] [2015-03-19 19:45:43,950] INFO >> {org.apache.stratos.messaging.message.processor.topology.ApplicationClustersRemovedMessageProcessor} >> - Cluster cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain removed >> from topology for application cisco-sample-vm >> TID: [0] [STRATOS] [2015-03-19 19:45:43,952] INFO >> {org.apache.stratos.messaging.message.processor.application.ApplicationDeletedMessageProcessor} >> - [Application] cisco-sample-vm has been successfully removed >> TID: [0] [STRATOS] [2015-03-19 19:45:44,786] INFO >> {org.apache.stratos.cloud.controller.iaases.JcloudsIaas} - Member >> terminated: [member-id] >> cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain41624077-30c4-4bfe-a2a8-ece2fc4f550d >> TID: [0] [STRATOS] [2015-03-19 19:45:44,786] WARN >> {org.apache.stratos.cloud.controller.messaging.topology.TopologyBuilder} - >> Cluster cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain does not >> exist >> TID: [0] [STRATOS] [2015-03-19 19:45:50,012] INFO >> {org.apache.stratos.common.client.CloudControllerServiceClient} - >> Terminating instance via cloud controller: [member] >> cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain41624077-30c4-4bfe-a2a8-ece2fc4f550d >> TID: [0] [STRATOS] [2015-03-19 19:45:50,017] ERROR >> {org.apache.stratos.cloud.controller.services.impl.CloudControllerServiceImpl} >> - Could not terminate instance, member context not found: [member-id] >> cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain41624077-30c4-4bfe-a2a8-ece2fc4f550d >> TID: [0] [STRATOS] [2015-03-19 19:45:50,018] ERROR >> {org.apache.stratos.cloud.controller.services.impl.CloudControllerServiceImpl} >> - Could not terminate instance: [member-id] >> cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain41624077-30c4-4bfe-a2a8-ece2fc4f550d >> org.apache.stratos.cloud.controller.exception.InvalidMemberException: >> Could not terminate instance, member context not found: [member-id] >> cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain41624077-30c4-4bfe-a2a8-ece2fc4f550d >> at >> org.apache.stratos.cloud.controller.services.impl.CloudControllerServiceImpl.terminateInstance(CloudControllerServiceImpl.java:553) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:606) >> at >> org.apache.axis2.rpc.receivers.RPCUtil.invokeServiceClass(RPCUtil.java:212) >> at >> org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver.invokeBusinessLogic(RPCInOnlyMessageReceiver.java:66) >> at >> org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:110) >> at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180) >> at >> org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:172) >> at >> org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:146) >> at >> org.wso2.carbon.core.transports.CarbonServlet.doPost(CarbonServlet.java:231) >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:755) >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:848) >> at >> org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61) >> at >> org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128) >> at >> org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68) >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:848) >> at >> org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68) >> at >> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305) >> at >> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) >> at >> org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:61) >> at >> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243) >> at >> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) >> at >> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222) >> at >> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123) >> at >> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472) >> at >> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171) >> at >> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99) >> at >> org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:178) >> at >> org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47) >> at >> org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:56) >> at >> org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47) >> at >> org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:141) >> at >> org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:156) >> at >> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936) >> at >> org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:52) >> at >> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118) >> at >> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407) >> at >> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004) >> at >> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589) >> at org.apache.tomcat.util.net >> .NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >> at java.lang.Thread.run(Thread.java:745) >> TID: [0] [STRATOS] [2015-03-19 19:45:50,019] WARN >> {org.apache.stratos.common.concurrent.locks.ReadWriteLock} - System >> warning! Trying to release a lock which has not been taken by the same >> thread: [lock-name] topology-manager [thread-id] 164 [thread-name] >> http-nio-9443-exec-29 >> TID: [0] [STRATOS] [2015-03-19 19:45:50,020] ERROR >> {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver} - Could not >> terminate instance, member context not found: [member-id] >> cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain41624077-30c4-4bfe-a2a8-ece2fc4f550d >> org.apache.stratos.cloud.controller.exception.InvalidMemberException: >> Could not terminate instance, member context not found: [member-id] >> cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain41624077-30c4-4bfe-a2a8-ece2fc4f550d >> at >> org.apache.stratos.cloud.controller.services.impl.CloudControllerServiceImpl.terminateInstance(CloudControllerServiceImpl.java:553) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:606) >> at >> org.apache.axis2.rpc.receivers.RPCUtil.invokeServiceClass(RPCUtil.java:212) >> at >> org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver.invokeBusinessLogic(RPCInOnlyMessageReceiver.java:66) >> at >> org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:110) >> at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180) >> at >> org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:172) >> at >> org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:146) >> at >> org.wso2.carbon.core.transports.CarbonServlet.doPost(CarbonServlet.java:231) >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:755) >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:848) >> at >> org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61) >> at >> org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128) >> at >> org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68) >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:848) >> at >> org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68) >> at >> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305) >> at >> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) >> at >> org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:61) >> at >> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243) >> at >> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) >> at >> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222) >> at >> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123) >> at >> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472) >> at >> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171) >> at >> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99) >> at >> org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:178) >> at >> org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47) >> at >> org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:56) >> at >> org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47) >> at >> org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:141) >> at >> org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:156) >> at >> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936) >> at >> org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:52) >> at >> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118) >> at >> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407) >> at >> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004) >> at >> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589) >> at org.apache.tomcat.util.net >> .NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >> at java.lang.Thread.run(Thread.java:745) >> TID: [0] [STRATOS] [2015-03-19 19:45:50,027] ERROR >> {org.apache.stratos.autoscaler.rule.RuleTasksDelegator} - Cannot terminate >> instance >> org.apache.axis2.AxisFault: Could not terminate instance, member context >> not found: [member-id] >> cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain41624077-30c4-4bfe-a2a8-ece2fc4f550d >> at >> org.apache.axis2.util.Utils.getInboundFaultFromMessageContext(Utils.java:531) >> at >> org.apache.axis2.description.RobustOutOnlyAxisOperation$RobustOutOnlyOperationClient.handleResponse(RobustOutOnlyAxisOperation.java:91) >> at >> org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:445) >> at >> org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:225) >> at >> org.apache.axis2.client.OperationClient.execute(OperationClient.java:149) >> at >> org.apache.stratos.cloud.controller.stub.CloudControllerServiceStub.terminateInstance(CloudControllerServiceStub.java:7071) >> at >> org.apache.stratos.common.client.CloudControllerServiceClient.terminateInstance(CloudControllerServiceClient.java:120) >> at >> org.apache.stratos.autoscaler.rule.RuleTasksDelegator.terminateObsoleteInstance(RuleTasksDelegator.java:311) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:606) >> at >> org.mvel2.optimizers.impl.refl.nodes.MethodAccessor.getValue(MethodAccessor.java:48) >> at >> org.mvel2.optimizers.impl.refl.nodes.VariableAccessor.getValue(VariableAccessor.java:37) >> at >> org.mvel2.ast.ASTNode.getReducedValueAccelerated(ASTNode.java:108) >> at org.mvel2.MVELRuntime.execute(MVELRuntime.java:85) >> at >> org.mvel2.compiler.CompiledExpression.getDirectValue(CompiledExpression.java:123) >> at >> org.mvel2.compiler.CompiledExpression.getValue(CompiledExpression.java:119) >> at org.mvel2.MVEL.executeExpression(MVEL.java:930) >> at >> org.drools.base.mvel.MVELConsequence.evaluate(MVELConsequence.java:104) >> at >> org.drools.common.DefaultAgenda.fireActivation(DefaultAgenda.java:1287) >> at >> org.drools.common.DefaultAgenda.fireNextItem(DefaultAgenda.java:1221) >> >> >> >> >> >> -- >> >> Imesh Gunaratne >> >> >> >> Technical Lead, WSO2 >> >> Committer & PMC Member, Apache Stratos >> > > > > -- > Imesh Gunaratne > > Technical Lead, WSO2 > Committer & PMC Member, Apache Stratos > -- Udara Liyanage Software Engineer WSO2, Inc.: http://wso2.com lean. enterprise. middleware web: http://udaraliyanage.wordpress.com phone: +94 71 443 6897
