[Openstack] Cleaning nova database

2012-03-20 Thread David Kranz
In a diablo/kvm cluster that has been running for a long time, a user reported problems with some vms, tried rebooting them and eventually deleted them. I recently noticed messages in the nova compute log like: Found 13 in the database and 10 on the hypervisor. Looking at the source code I

Re: [Openstack] Cleaning nova database

2012-03-20 Thread Leandro Reox
I think that the quick solution is set deteled to 1 on the offending instances Are u using euca tools ? some inconsistencies are generated by them often Regards Lean On Tue, Mar 20, 2012 at 12:19 PM, David Kranz david.kr...@qrclab.comwrote: In a diablo/kvm cluster that has been running for a

Re: [Openstack] Cleaning nova database

2012-03-20 Thread Jay Pipes
On 03/20/2012 11:19 AM, David Kranz wrote: In a diablo/kvm cluster that has been running for a long time, a user reported problems with some vms, tried rebooting them and eventually deleted them. I recently noticed messages in the nova compute log like: Found 13 in the database and 10 on the

Re: [Openstack] Cleaning nova database

2012-03-20 Thread David Kranz
The users are using nova CLI, not euca. The 'deleted' field is already 1. The delete fails because the id is a foreign key in the virtual_interfaces table. The question is how to excise an instance from the database without screwing anything up. Here is the whole row, which has a few

Re: [Openstack] Cleaning nova database

2012-03-20 Thread David Kranz
Thanks, Jay. I agree with your comments about Essex. The problem is that I have cleaned up after a number of operational problems in a cluster that has been up for 3 months. It is hard to reproduce these problems when the mean time to failure is so long, and real investigation can be dangerous