Public bug reported:
[Description]
Occasionally when evacuating vms off of nova compute hosts for host reboots, a
vms migration will be reported as complete in the migration list, but queries
to the openstack api, such as 'openstack show uuid' will report the host &
hypervisor-hostname unchanged. The only indication that something is wrong is
that power_state will be NOSTATE. We can see that the instance is in fact
migrated and running on the new host with 'sudo virsh list --all | grep
$instance_name'.
In order to resolve this issue we perform a direct database edit such
as:
'update instances
set host="$newhost", node="$newhost.domain", progress="0"
where uuid="" and deleted="0";'
* In one instance, the 'progress' value was stuck at 99 and I needed to
set that to 0 in the database as well.
[Expected]
Its expected that the live migration completes and that the instance in the
openstack database correctly reflects the name of the new host, and its power
state.
[Impact]
Instances that are found to be in power state NOSTATE are blocked from
performing certain actions; instances in this state do not self recover.
[Environment]
Openstack Queens; Nova 17.0.10
libvirtd/virsh: 4.0.0
ceph: 12.2.8
neutron-openvswitch: 12.0.5
[Logs]
In this particular set of logs (sosreports from the live migration source and
destination hosts); the instance that was in error had uuid
67f328d0-cb5e-416a-9af4-c6e47e68a1e0.
** Affects: nova
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1849154
Title:
Live Migrations complete but occasionally fail to update the Openstack
Database
Status in OpenStack Compute (nova):
New
Bug description:
[Description]
Occasionally when evacuating vms off of nova compute hosts for host reboots,
a vms migration will be reported as complete in the migration list, but queries
to the openstack api, such as 'openstack show uuid' will report the host &
hypervisor-hostname unchanged. The only indication that something is wrong is
that power_state will be NOSTATE. We can see that the instance is in fact
migrated and running on the new host with 'sudo virsh list --all | grep
$instance_name'.
In order to resolve this issue we perform a direct database edit such
as:
'update instances
set host="$newhost", node="$newhost.domain", progress="0"
where uuid="" and deleted="0";'
* In one instance, the 'progress' value was stuck at 99 and I needed
to set that to 0 in the database as well.
[Expected]
Its expected that the live migration completes and that the instance in the
openstack database correctly reflects the name of the new host, and its power
state.
[Impact]
Instances that are found to be in power state NOSTATE are blocked from
performing certain actions; instances in this state do not self recover.
[Environment]
Openstack Queens; Nova 17.0.10
libvirtd/virsh: 4.0.0
ceph: 12.2.8
neutron-openvswitch: 12.0.5
[Logs]
In this particular set of logs (sosreports from the live migration source and
destination hosts); the instance that was in error had uuid
67f328d0-cb5e-416a-9af4-c6e47e68a1e0.
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1849154/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
