Public bug reported: When e.g. nova-compute or rabbit-server dies during live migration and somehow nova-compute is not able to report new host for migrated VM, then after successful system recovery nova deletes VM instead of sending host update. This is from nova log:
09:00:25.704 INFO nova.compute.manager [-] [instance: b8a3bdd6-809f-44b4-875d-df3feafab41a] Deleting instance as its host (node-16) is not equal to our host (node-15). 09:00:27.972 INFO oslo.messaging._drivers.impl_rabbit [-] Reconnecting to AMQP server on 10.4.8.2:5672 09:00:27.972 INFO oslo.messaging._drivers.impl_rabbit [-] Delaying reconnect for 1.0 seconds... 09:00:28.981 INFO oslo.messaging._drivers.impl_rabbit [-] Connected to AMQP server on 10.4.8.2:5672 09:00:36.464 INFO nova.compute.manager [-] Lifecycle event 1 on VM b8a3bdd6-809f-44b4-875d-df3feafab41a 09:00:36.468 INFO nova.virt.libvirt.driver [-] [instance: b8a3bdd6-809f-44b4-875d-df3feafab41a] Instance destroyed successfully. 09:00:36.471 INFO nova.virt.libvirt.firewall [-] [instance: b8a3bdd6-809f-44b4-875d-df3feafab41a] Attempted to unfilter instance which is not filtered 09:00:36.521 INFO oslo.messaging._drivers.impl_rabbit [-] Connected to AMQP server on 10.4.8.2:5672 09:00:36.565 INFO nova.compute.manager [req-93e15eda-8d65-49f5-a195-52b91da7aa68 None] [instance: b8a3bdd6-809f-44b4-875d-df3feafab41a] During the sync_power process the instance has moved from host node-15 to host node-16 09:00:36.566 INFO nova.virt.libvirt.driver [-] [instance: b8a3bdd6-809f-44b4-875d-df3feafab41a] Deleting instance files /var/lib/nova/instances/b8a3bdd6-809f-44b4-875d-df3feafab41a 09:00:36.566 INFO nova.virt.libvirt.driver [-] [instance: b8a3bdd6-809f-44b4-875d-df3feafab41a] Deletion of /var/lib/nova/instances/b8a3bdd6-809f-44b4-875d-df3feafab41a complete However VM record in database is still present (with state MIGRATING) and volume is still attached to VM that does not exist. ** Affects: nova Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1396529 Title: Nova deletes instance when compute/rabbit is dead at the end of live migration Status in OpenStack Compute (Nova): New Bug description: When e.g. nova-compute or rabbit-server dies during live migration and somehow nova-compute is not able to report new host for migrated VM, then after successful system recovery nova deletes VM instead of sending host update. This is from nova log: 09:00:25.704 INFO nova.compute.manager [-] [instance: b8a3bdd6-809f-44b4-875d-df3feafab41a] Deleting instance as its host (node-16) is not equal to our host (node-15). 09:00:27.972 INFO oslo.messaging._drivers.impl_rabbit [-] Reconnecting to AMQP server on 10.4.8.2:5672 09:00:27.972 INFO oslo.messaging._drivers.impl_rabbit [-] Delaying reconnect for 1.0 seconds... 09:00:28.981 INFO oslo.messaging._drivers.impl_rabbit [-] Connected to AMQP server on 10.4.8.2:5672 09:00:36.464 INFO nova.compute.manager [-] Lifecycle event 1 on VM b8a3bdd6-809f-44b4-875d-df3feafab41a 09:00:36.468 INFO nova.virt.libvirt.driver [-] [instance: b8a3bdd6-809f-44b4-875d-df3feafab41a] Instance destroyed successfully. 09:00:36.471 INFO nova.virt.libvirt.firewall [-] [instance: b8a3bdd6-809f-44b4-875d-df3feafab41a] Attempted to unfilter instance which is not filtered 09:00:36.521 INFO oslo.messaging._drivers.impl_rabbit [-] Connected to AMQP server on 10.4.8.2:5672 09:00:36.565 INFO nova.compute.manager [req-93e15eda-8d65-49f5-a195-52b91da7aa68 None] [instance: b8a3bdd6-809f-44b4-875d-df3feafab41a] During the sync_power process the instance has moved from host node-15 to host node-16 09:00:36.566 INFO nova.virt.libvirt.driver [-] [instance: b8a3bdd6-809f-44b4-875d-df3feafab41a] Deleting instance files /var/lib/nova/instances/b8a3bdd6-809f-44b4-875d-df3feafab41a 09:00:36.566 INFO nova.virt.libvirt.driver [-] [instance: b8a3bdd6-809f-44b4-875d-df3feafab41a] Deletion of /var/lib/nova/instances/b8a3bdd6-809f-44b4-875d-df3feafab41a complete However VM record in database is still present (with state MIGRATING) and volume is still attached to VM that does not exist. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1396529/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp