Public bug reported:
When e.g. nova-compute or rabbit-server dies during live migration and
somehow nova-compute is not able to report new host for migrated VM,
then after successful system recovery nova deletes VM instead of sending
host update. This is from nova log:
09:00:25.704 INFO nova.compute.manager [-] [instance:
b8a3bdd6-809f-44b4-875d-df3feafab41a] Deleting instance as its host (node-16)
is not equal to our host (node-15).
09:00:27.972 INFO oslo.messaging._drivers.impl_rabbit [-] Reconnecting to AMQP
server on 10.4.8.2:5672
09:00:27.972 INFO oslo.messaging._drivers.impl_rabbit [-] Delaying reconnect
for 1.0 seconds...
09:00:28.981 INFO oslo.messaging._drivers.impl_rabbit [-] Connected to AMQP
server on 10.4.8.2:5672
09:00:36.464 INFO nova.compute.manager [-] Lifecycle event 1 on VM
b8a3bdd6-809f-44b4-875d-df3feafab41a
09:00:36.468 INFO nova.virt.libvirt.driver [-] [instance:
b8a3bdd6-809f-44b4-875d-df3feafab41a] Instance destroyed successfully.
09:00:36.471 INFO nova.virt.libvirt.firewall [-] [instance:
b8a3bdd6-809f-44b4-875d-df3feafab41a] Attempted to unfilter instance which is
not filtered
09:00:36.521 INFO oslo.messaging._drivers.impl_rabbit [-] Connected to AMQP
server on 10.4.8.2:5672
09:00:36.565 INFO nova.compute.manager
[req-93e15eda-8d65-49f5-a195-52b91da7aa68 None] [instance:
b8a3bdd6-809f-44b4-875d-df3feafab41a] During the sync_power process the
instance has moved from host node-15 to host node-16
09:00:36.566 INFO nova.virt.libvirt.driver [-] [instance:
b8a3bdd6-809f-44b4-875d-df3feafab41a] Deleting instance files
/var/lib/nova/instances/b8a3bdd6-809f-44b4-875d-df3feafab41a
09:00:36.566 INFO nova.virt.libvirt.driver [-] [instance:
b8a3bdd6-809f-44b4-875d-df3feafab41a] Deletion of
/var/lib/nova/instances/b8a3bdd6-809f-44b4-875d-df3feafab41a complete
However VM record in database is still present (with state MIGRATING)
and volume is still attached to VM that does not exist.
** Affects: nova
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1396529
Title:
Nova deletes instance when compute/rabbit is dead at the end of live
migration
Status in OpenStack Compute (Nova):
New
Bug description:
When e.g. nova-compute or rabbit-server dies during live migration and
somehow nova-compute is not able to report new host for migrated VM,
then after successful system recovery nova deletes VM instead of
sending host update. This is from nova log:
09:00:25.704 INFO nova.compute.manager [-] [instance:
b8a3bdd6-809f-44b4-875d-df3feafab41a] Deleting instance as its host (node-16)
is not equal to our host (node-15).
09:00:27.972 INFO oslo.messaging._drivers.impl_rabbit [-] Reconnecting to
AMQP server on 10.4.8.2:5672
09:00:27.972 INFO oslo.messaging._drivers.impl_rabbit [-] Delaying reconnect
for 1.0 seconds...
09:00:28.981 INFO oslo.messaging._drivers.impl_rabbit [-] Connected to AMQP
server on 10.4.8.2:5672
09:00:36.464 INFO nova.compute.manager [-] Lifecycle event 1 on VM
b8a3bdd6-809f-44b4-875d-df3feafab41a
09:00:36.468 INFO nova.virt.libvirt.driver [-] [instance:
b8a3bdd6-809f-44b4-875d-df3feafab41a] Instance destroyed successfully.
09:00:36.471 INFO nova.virt.libvirt.firewall [-] [instance:
b8a3bdd6-809f-44b4-875d-df3feafab41a] Attempted to unfilter instance which is
not filtered
09:00:36.521 INFO oslo.messaging._drivers.impl_rabbit [-] Connected to AMQP
server on 10.4.8.2:5672
09:00:36.565 INFO nova.compute.manager
[req-93e15eda-8d65-49f5-a195-52b91da7aa68 None] [instance:
b8a3bdd6-809f-44b4-875d-df3feafab41a] During the sync_power process the
instance has moved from host node-15 to host node-16
09:00:36.566 INFO nova.virt.libvirt.driver [-] [instance:
b8a3bdd6-809f-44b4-875d-df3feafab41a] Deleting instance files
/var/lib/nova/instances/b8a3bdd6-809f-44b4-875d-df3feafab41a
09:00:36.566 INFO nova.virt.libvirt.driver [-] [instance:
b8a3bdd6-809f-44b4-875d-df3feafab41a] Deletion of
/var/lib/nova/instances/b8a3bdd6-809f-44b4-875d-df3feafab41a complete
However VM record in database is still present (with state MIGRATING)
and volume is still attached to VM that does not exist.
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1396529/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
