Hi Anuruddha, The instances that are in Error state on Openstack Horizon were never removed even after Stratos successfully spawned an instance. It sounds like you might need to enhance jClouds API to return an object with nodeID info for this case. Or perhaps a better solution would be to modify the jClouds API to delete the failed instance if it wasn’t spawned successfully (or make that an option of the API that handles spawning new instance).
Regards, -Jeffrey From: Anuruddha Liyanarachchi <[email protected]<mailto:[email protected]>> Reply-To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Date: Thursday, April 2, 2015 at 4:43 AM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: Re: Stratos not properly terminating VMs to fail to startup Hi Vanson / Jeffery, As seen in logs, the instance Id is not returned to Stratos (instanceId=null) for the members which went to error state.Therefore Stratos don't have control over the instances in the error state. Hence spawned instances with errors are not being deleted. On Wed, Apr 1, 2015 at 4:08 AM, Jeffrey Nguyen (jeffrngu) <[email protected]<mailto:[email protected]>> wrote: Hi Vanson, I opened a JIRA to track this issue last week: https://issues.apache.org/jira/browse/STRATOS-1293 -Jeffrey On 3/31/15, 3:04 PM, "Vanson Lim (vlim)" <[email protected]<mailto:[email protected]>> wrote: >Devs, > >I've simulated the case where openstack fails to bring up a VM (we've >seen this before in cases where required resources are not available >or there is some IAAS problem/timeout which caused the VM to failure to >launch), in this case we cause this failure by specifying the >cartridge to have a fixed ip address is not part of the network which the >VM attaches to. The network is defined with a 10.0.0.0/24<http://10.0.0.0/24> >subnet, but I've specified a fixed ip=10.0.8.1 for cause the VM startup >to fail. > >The VM start fails and the VM remains in an error state the of the >"pendingMemberExpiryTimeout" period set in the autoscaler.xml file. > >Stratos fails to delete the VM in error state and attempts to start a new >VM, which also fails to launch. > >This presumably repeat itself creating an additional VM in error state >during each iteration until we've exhausted all the resources in the >system. > >wso2carbon.log and cartridge definition attached. > >-Vanson > -- Thanks and Regards, Anuruddha Lanka Liyanarachchi Software Engineer - WSO2 Mobile : +94 (0) 712762611 Tel : +94 112 145 345 a<mailto:[email protected]>[email protected]<mailto:[email protected]>
