Thanks Martin! We will have a look at this. On Tue, Mar 24, 2015 at 1:21 AM, Martin Eppel (meppel) <mep...@cisco.com> wrote:
> I opened a jira https://issues.apache.org/jira/browse/STRATOS-1281 to > track the issue, > > > > Thanks > > > > Martin > > > > *From:* Imesh Gunaratne [mailto:im...@apache.org] > *Sent:* Friday, March 20, 2015 3:27 AM > *To:* dev > *Subject:* Re: Stratos 4.1.0 - tracebacks seen when issuing application > undeploy/remove > > > > Hi Vanson, > > > > Thanks for reporting this problem. According to the logs stratos is trying > to remove the same member twice in this flow: > > > > TID: [0] [STRATOS] [2015-03-19 19:45:43,952] INFO > {org.apache.stratos.messaging.message.processor.application.ApplicationDeletedMessageProcessor} > - [Application] cisco-sample-vm has been successfully removed > > TID: [0] [STRATOS] [2015-03-19 19:45:44,786] INFO > {org.apache.stratos.cloud.controller.iaases.JcloudsIaas} - Member > terminated: [member-id] > cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain41624077-30c4-4bfe-a2a8-ece2fc4f550d > > > > TID: [0] [STRATOS] [2015-03-19 19:45:50,012] INFO > {org.apache.stratos.common.client.CloudControllerServiceClient} - > Terminating instance via cloud controller: [member] > cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain41624077-30c4-4bfe-a2a8-ece2fc4f550d > > TID: [0] [STRATOS] [2015-03-19 19:45:50,017] ERROR > {org.apache.stratos.cloud.controller.services.impl.CloudControllerServiceImpl} > - Could not terminate instance, member context not found: [member-id] > cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain41624077-30c4-4bfe-a2a8-ece2fc4f550d > > > > We might need to investigate and see what's causing this. > > > > Thanks > > > > > > On Fri, Mar 20, 2015 at 6:55 AM, Vanson Lim <v...@cisco.com> wrote: > > Hi, > > We are testing the behavior of stratos 4.1.0 rest api's and found that if > we issue an undeploy-application followed immediately with > application-remove, that we see the following traceback. > > The VM successfully get's deleted but not sure what kind of side effects > this has on the system. > > Here's the snippet from the wso2carbon.log file. I've also attached the > entire log. > > Steps to reproduce this: > > 1) deploy an application which leads to startup a single instance of a > cartridge. > 2) wait for it to become active > 3) issue the undeploy application and application remove rest api calls. > > > -Vanson > > > > TID: [0] [STRATOS] [2015-03-19 19:45:08,681] INFO > {org.wso2.carbon.databridge.core.DataBridge} - admin connected > TID: [0] [STRATOS] [2015-03-19 19:45:43,023] INFO > {org.apache.stratos.autoscaler.services.impl.AutoscalerServiceImpl} - > Starting to undeploy application: [application-id] cisco-sample-vm > TID: [0] [STRATOS] [2015-03-19 19:45:43,024] INFO > {org.apache.stratos.autoscaler.services.impl.AutoscalerServiceImpl} - > Removing application signup: [application-id] cisco-sample-vm > TID: [0] [STRATOS] [2015-03-19 19:45:43,051] INFO > {org.apache.stratos.manager.components.ApplicationSignUpHandler} - Removing > application signup: [application-id] cisco-sample-vm [tenant-id] -1234 > TID: [0] [STRATOS] [2015-03-19 19:45:43,084] INFO > {org.apache.stratos.manager.components.ApplicationSignUpHandler} - > Application signup removed successfully: [application-id] cisco-sample-vm > [tenant-id] -1234 > TID: [0] [STRATOS] [2015-03-19 19:45:43,091] INFO > {org.apache.stratos.autoscaler.context.AutoscalerContext} - Network > partition algorithm context is removed successfully: [id] cisco-sample-vm > TID: [0] [STRATOS] [2015-03-19 19:45:43,092] INFO > {org.apache.stratos.autoscaler.monitor.cluster.ClusterMonitor} - Publishing > Cluster terminating event for [application] cisco-sample-vm [cluster] > cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain [instance] > cisco-sample-vm-1 > TID: [0] [STRATOS] [2015-03-19 19:45:43,109] INFO > {org.apache.stratos.cloud.controller.messaging.topology.TopologyBuilder} - > Cluster Terminating adding status started > forcisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain > TID: [0] [STRATOS] [2015-03-19 19:45:43,117] INFO > {org.apache.stratos.autoscaler.applications.topic.ApplicationsEventPublisher} > - Publishing application inactivated event: [application] cisco-sample-vm > [instance] cisco-sample-vm-1 > TID: [0] [STRATOS] [2015-03-19 19:45:43,120] INFO > {org.apache.stratos.cloud.controller.messaging.publisher.TopologyEventPublisher} > - Publishing Cluster terminating event: [application-id] cisco-sample-vm > [cluster id] cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain > [instance-id] cisco-sample-vm-1 > TID: [0] [STRATOS] [2015-03-19 19:45:43,144] INFO > {org.apache.stratos.autoscaler.services.impl.AutoscalerServiceImpl} - > Application undeployed successfully: [application-id] cisco-sample-vm > TID: [0] [STRATOS] [2015-03-19 19:45:43,154] INFO > {org.apache.stratos.autoscaler.event.receiver.topology.AutoscalerTopologyEventReceiver} > - [ClusterTerminatingEvent] Received: class > org.apache.stratos.messaging.event.topology.ClusterInstanceTerminatingEvent > TID: [0] [STRATOS] [2015-03-19 19:45:43,155] INFO > {org.apache.stratos.autoscaler.event.publisher.InstanceNotificationPublisher} > - Publishing Instance Cleanup Event: [cluster] > cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain > TID: [0] [STRATOS] [2015-03-19 19:45:43,171] WARN > {org.apache.stratos.autoscaler.status.processor.cluster.ClusterStatusActiveProcessor} > - No possible state change found for [type] [cluster] > cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain [instance] > TID: [0] [STRATOS] [2015-03-19 19:45:43,192] INFO > {org.apache.stratos.cloud.controller.messaging.topology.TopologyBuilder} - > member maintenance mode event adding status started > TID: [0] [STRATOS] [2015-03-19 19:45:43,204] INFO > {org.apache.stratos.cloud.controller.messaging.publisher.TopologyEventPublisher} > - Publishing member in maintenance mode event: [service-name] > cisco-sample-vm [cluster-id] > cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain > [cluster-instance-id] cisco-sample-vm-1 [member-id] > cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain41624077-30c4-4bfe-a2a8-ece2fc4f550d > [network-partition-id] RegionOne [partition-id] whole-region > TID: [0] [STRATOS] [2015-03-19 19:45:43,230] INFO > {org.apache.stratos.messaging.message.processor.topology.MemberMaintenanceModeProcessor} > - Member updated as In_Maintenance: [service] cisco-sample-vm [cluster] > cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain [member] > cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain41624077-30c4-4bfe-a2a8-ece2fc4f550d > TID: [0] [STRATOS] [2015-03-19 19:45:43,231] INFO > {org.apache.stratos.cloud.controller.messaging.topology.TopologyBuilder} - > Member Ready to shut down event adding status started > TID: [0] [STRATOS] [2015-03-19 19:45:43,242] INFO > {org.apache.stratos.cloud.controller.messaging.publisher.TopologyEventPublisher} > - Publishing member ready to shut down event: [service-name] > cisco-sample-vm [cluster-id] > cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain > [cluster-instance-id] cisco-sample-vm-1 [member-id] > cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain41624077-30c4-4bfe-a2a8-ece2fc4f550d > [network-partition-id] RegionOne [partition-id] whole-region > TID: [0] [STRATOS] [2015-03-19 19:45:43,262] INFO > {org.apache.stratos.messaging.message.processor.topology.MemberReadyToShutdownMessageProcessor} > - Member updated as Ready to shutdown: [service] cisco-sample-vm [cluster] > cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain [member] > cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain41624077-30c4-4bfe-a2a8-ece2fc4f550d > TID: [0] [STRATOS] [2015-03-19 19:45:43,264] INFO > {org.apache.stratos.autoscaler.event.receiver.topology.AutoscalerTopologyEventReceiver} > - [MemberReadyToShutdownEvent] Received: class > org.apache.stratos.messaging.event.topology.MemberReadyToShutdownEvent > TID: [0] [STRATOS] [2015-03-19 19:45:43,264] INFO > {org.apache.stratos.autoscaler.context.partition.ClusterLevelPartitionContext} > - Starting the moving of termination pending to obsolete for [member] > cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain41624077-30c4-4bfe-a2a8-ece2fc4f550d > TID: [0] [STRATOS] [2015-03-19 19:45:43,264] INFO > {org.apache.stratos.autoscaler.context.partition.ClusterLevelPartitionContext} > - Found termination pending member and trying to move [member] > cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain41624077-30c4-4bfe-a2a8-ece2fc4f550d > to obsolete list > TID: [0] [STRATOS] [2015-03-19 19:45:43,266] INFO > {org.apache.stratos.common.client.CloudControllerServiceClient} - > Terminating instance via cloud controller: [member] > cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain41624077-30c4-4bfe-a2a8-ece2fc4f550d > TID: [0] [STRATOS] [2015-03-19 19:45:43,275] INFO > {org.apache.stratos.cloud.controller.iaases.JcloudsIaas} - Starting to > terminate member: [cartridge-type] cisco-sample-vm [member-id] > cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain41624077-30c4-4bfe-a2a8-ece2fc4f550d > TID: [0] [STRATOS] [2015-03-19 19:45:43,883] INFO > {org.apache.stratos.messaging.domain.application.locking.ApplicationLockHierarchy} > - Removed lock for application: [application-id] cisco-sample-vm > TID: [0] [STRATOS] [2015-03-19 19:45:43,909] INFO > {org.apache.stratos.cloud.controller.messaging.receiver.application.ApplicationEventReceiver} > - Application deleted event received: [application-id] cisco-sample-vm > TID: [0] [STRATOS] [2015-03-19 19:45:43,909] INFO > {org.apache.stratos.cloud.controller.messaging.topology.TopologyBuilder} - > Removed application [ cisco-sample-vm ]'s Cluster [ > cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain ] from the topology > TID: [0] [STRATOS] [2015-03-19 19:45:43,910] INFO > {org.apache.stratos.autoscaler.services.impl.AutoscalerServiceImpl} - > Application deleted successfully: [application-id] > TID: [0] [STRATOS] [2015-03-19 19:45:43,930] INFO > {org.apache.stratos.cloud.controller.messaging.publisher.TopologyEventPublisher} > - Publishing application clusters removed event: [application-id] > cisco-sample-vm > TID: [0] [STRATOS] [2015-03-19 19:45:43,950] INFO > {org.apache.stratos.messaging.message.processor.topology.ApplicationClustersRemovedMessageProcessor} > - Cluster cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain removed > from topology for application cisco-sample-vm > TID: [0] [STRATOS] [2015-03-19 19:45:43,952] INFO > {org.apache.stratos.messaging.message.processor.application.ApplicationDeletedMessageProcessor} > - [Application] cisco-sample-vm has been successfully removed > TID: [0] [STRATOS] [2015-03-19 19:45:44,786] INFO > {org.apache.stratos.cloud.controller.iaases.JcloudsIaas} - Member > terminated: [member-id] > cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain41624077-30c4-4bfe-a2a8-ece2fc4f550d > TID: [0] [STRATOS] [2015-03-19 19:45:44,786] WARN > {org.apache.stratos.cloud.controller.messaging.topology.TopologyBuilder} - > Cluster cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain does not > exist > TID: [0] [STRATOS] [2015-03-19 19:45:50,012] INFO > {org.apache.stratos.common.client.CloudControllerServiceClient} - > Terminating instance via cloud controller: [member] > cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain41624077-30c4-4bfe-a2a8-ece2fc4f550d > TID: [0] [STRATOS] [2015-03-19 19:45:50,017] ERROR > {org.apache.stratos.cloud.controller.services.impl.CloudControllerServiceImpl} > - Could not terminate instance, member context not found: [member-id] > cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain41624077-30c4-4bfe-a2a8-ece2fc4f550d > TID: [0] [STRATOS] [2015-03-19 19:45:50,018] ERROR > {org.apache.stratos.cloud.controller.services.impl.CloudControllerServiceImpl} > - Could not terminate instance: [member-id] > cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain41624077-30c4-4bfe-a2a8-ece2fc4f550d > org.apache.stratos.cloud.controller.exception.InvalidMemberException: > Could not terminate instance, member context not found: [member-id] > cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain41624077-30c4-4bfe-a2a8-ece2fc4f550d > at > org.apache.stratos.cloud.controller.services.impl.CloudControllerServiceImpl.terminateInstance(CloudControllerServiceImpl.java:553) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > org.apache.axis2.rpc.receivers.RPCUtil.invokeServiceClass(RPCUtil.java:212) > at > org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver.invokeBusinessLogic(RPCInOnlyMessageReceiver.java:66) > at > org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:110) > at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180) > at > org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:172) > at > org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:146) > at > org.wso2.carbon.core.transports.CarbonServlet.doPost(CarbonServlet.java:231) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:755) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:848) > at > org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61) > at > org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128) > at > org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:848) > at > org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) > at > org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:61) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) > at > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222) > at > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123) > at > org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472) > at > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171) > at > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99) > at > org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:178) > at > org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47) > at > org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:56) > at > org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47) > at > org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:141) > at > org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:156) > at > org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936) > at > org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:52) > at > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118) > at > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407) > at > org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004) > at > org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589) > at org.apache.tomcat.util.net > .NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > TID: [0] [STRATOS] [2015-03-19 19:45:50,019] WARN > {org.apache.stratos.common.concurrent.locks.ReadWriteLock} - System > warning! Trying to release a lock which has not been taken by the same > thread: [lock-name] topology-manager [thread-id] 164 [thread-name] > http-nio-9443-exec-29 > TID: [0] [STRATOS] [2015-03-19 19:45:50,020] ERROR > {org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver} - Could not > terminate instance, member context not found: [member-id] > cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain41624077-30c4-4bfe-a2a8-ece2fc4f550d > org.apache.stratos.cloud.controller.exception.InvalidMemberException: > Could not terminate instance, member context not found: [member-id] > cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain41624077-30c4-4bfe-a2a8-ece2fc4f550d > at > org.apache.stratos.cloud.controller.services.impl.CloudControllerServiceImpl.terminateInstance(CloudControllerServiceImpl.java:553) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > org.apache.axis2.rpc.receivers.RPCUtil.invokeServiceClass(RPCUtil.java:212) > at > org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver.invokeBusinessLogic(RPCInOnlyMessageReceiver.java:66) > at > org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:110) > at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180) > at > org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:172) > at > org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:146) > at > org.wso2.carbon.core.transports.CarbonServlet.doPost(CarbonServlet.java:231) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:755) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:848) > at > org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61) > at > org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128) > at > org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:848) > at > org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) > at > org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:61) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) > at > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222) > at > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123) > at > org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472) > at > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171) > at > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99) > at > org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:178) > at > org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47) > at > org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:56) > at > org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47) > at > org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:141) > at > org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:156) > at > org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936) > at > org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:52) > at > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118) > at > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407) > at > org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004) > at > org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589) > at org.apache.tomcat.util.net > .NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > TID: [0] [STRATOS] [2015-03-19 19:45:50,027] ERROR > {org.apache.stratos.autoscaler.rule.RuleTasksDelegator} - Cannot terminate > instance > org.apache.axis2.AxisFault: Could not terminate instance, member context > not found: [member-id] > cisco-sample-vm.cisco-sample-vm.cisco-sample-vm.domain41624077-30c4-4bfe-a2a8-ece2fc4f550d > at > org.apache.axis2.util.Utils.getInboundFaultFromMessageContext(Utils.java:531) > at > org.apache.axis2.description.RobustOutOnlyAxisOperation$RobustOutOnlyOperationClient.handleResponse(RobustOutOnlyAxisOperation.java:91) > at > org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:445) > at > org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:225) > at > org.apache.axis2.client.OperationClient.execute(OperationClient.java:149) > at > org.apache.stratos.cloud.controller.stub.CloudControllerServiceStub.terminateInstance(CloudControllerServiceStub.java:7071) > at > org.apache.stratos.common.client.CloudControllerServiceClient.terminateInstance(CloudControllerServiceClient.java:120) > at > org.apache.stratos.autoscaler.rule.RuleTasksDelegator.terminateObsoleteInstance(RuleTasksDelegator.java:311) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > org.mvel2.optimizers.impl.refl.nodes.MethodAccessor.getValue(MethodAccessor.java:48) > at > org.mvel2.optimizers.impl.refl.nodes.VariableAccessor.getValue(VariableAccessor.java:37) > at > org.mvel2.ast.ASTNode.getReducedValueAccelerated(ASTNode.java:108) > at org.mvel2.MVELRuntime.execute(MVELRuntime.java:85) > at > org.mvel2.compiler.CompiledExpression.getDirectValue(CompiledExpression.java:123) > at > org.mvel2.compiler.CompiledExpression.getValue(CompiledExpression.java:119) > at org.mvel2.MVEL.executeExpression(MVEL.java:930) > at > org.drools.base.mvel.MVELConsequence.evaluate(MVELConsequence.java:104) > at > org.drools.common.DefaultAgenda.fireActivation(DefaultAgenda.java:1287) > at > org.drools.common.DefaultAgenda.fireNextItem(DefaultAgenda.java:1221) > > > > > > -- > > Imesh Gunaratne > > > > Technical Lead, WSO2 > > Committer & PMC Member, Apache Stratos > -- Imesh Gunaratne Technical Lead, WSO2 Committer & PMC Member, Apache Stratos
