[Yahoo-eng-team] [Bug 1321082] Re: libvirt driver detach_volume fails after migration failure
** Changed in: nova Status: Fix Committed = Fix Released ** Changed in: nova Milestone: None = juno-1 -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1321082 Title: libvirt driver detach_volume fails after migration failure Status in OpenStack Compute (Nova): Fix Released Status in OpenStack Compute (nova) icehouse series: Fix Released Bug description: When a VM with an attached iSCSI disk fails to migrate, the rollback methods does not detach the disk from target host. What happens is _lookup_by_name() fails, since the VM does not exist on the target host. In detach_volume(), it is supposed to print a warning based on the correct error code being returned, instead of throwing the exception. However, this is not happening, because _lookup_by_name() throws an InstanceNotFound exception, rather than a libvirt.libvirtError exception. So we also need to catch InstanceNotFound exception, so that detach_volume() can continue to execute as expected. Here's the exception log that I have: 2014-05-16 16:30:22.328 41419 WARNING nova.compute.manager [req-3db28fed-c287-4b41-ac95-9a37a619c75c 0 4be9915c10c8426cbfe948940f7c8af1] [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c] Detaching volume from unknown instance 2014-05-16 16:30:22.331 41419 ERROR nova.compute.manager [req-3db28fed-c287-4b41-ac95-9a37a619c75c 0 4be9915c10c8426cbfe948940f7c8af1] [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c] Failed to detach volume 98a940e5-051f-4d0f-a8c7-859a5079d95e from /dev/vdb 2014-05-16 16:30:22.331 41419 TRACE nova.compute.manager [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c] Traceback (most recent call last): 2014-05-16 16:30:22.331 41419 TRACE nova.compute.manager [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c] File /usr/lib/python2.7/site-packages/nova/compute/manager.py, line 4218, in _detach_volume 2014-05-16 16:30:22.331 41419 TRACE nova.compute.manager [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c] encryption=encryption) 2014-05-16 16:30:22.331 41419 TRACE nova.compute.manager [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c] File /usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py, line 1356, in detach_volume 2014-05-16 16:30:22.331 41419 TRACE nova.compute.manager [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c] virt_dom = self._lookup_by_name(instance_name) 2014-05-16 16:30:22.331 41419 TRACE nova.compute.manager [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c] File /usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py, line 3477, in _lookup_by_name 2014-05-16 16:30:22.331 41419 TRACE nova.compute.manager [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c] raise exception.InstanceNotFound(instance_id=instance_name) 2014-05-16 16:30:22.331 41419 TRACE nova.compute.manager [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c] InstanceNotFound: Instance rhel65_113-3e1d0d56-0002 could not be found. 2014-05-16 16:30:22.331 41419 TRACE nova.compute.manager [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c] To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1321082/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1321082] Re: libvirt driver detach_volume fails after migration failure
** Changed in: nova/icehouse Status: Fix Committed = Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1321082 Title: libvirt driver detach_volume fails after migration failure Status in OpenStack Compute (Nova): Fix Committed Status in OpenStack Compute (nova) icehouse series: Fix Released Bug description: When a VM with an attached iSCSI disk fails to migrate, the rollback methods does not detach the disk from target host. What happens is _lookup_by_name() fails, since the VM does not exist on the target host. In detach_volume(), it is supposed to print a warning based on the correct error code being returned, instead of throwing the exception. However, this is not happening, because _lookup_by_name() throws an InstanceNotFound exception, rather than a libvirt.libvirtError exception. So we also need to catch InstanceNotFound exception, so that detach_volume() can continue to execute as expected. Here's the exception log that I have: 2014-05-16 16:30:22.328 41419 WARNING nova.compute.manager [req-3db28fed-c287-4b41-ac95-9a37a619c75c 0 4be9915c10c8426cbfe948940f7c8af1] [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c] Detaching volume from unknown instance 2014-05-16 16:30:22.331 41419 ERROR nova.compute.manager [req-3db28fed-c287-4b41-ac95-9a37a619c75c 0 4be9915c10c8426cbfe948940f7c8af1] [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c] Failed to detach volume 98a940e5-051f-4d0f-a8c7-859a5079d95e from /dev/vdb 2014-05-16 16:30:22.331 41419 TRACE nova.compute.manager [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c] Traceback (most recent call last): 2014-05-16 16:30:22.331 41419 TRACE nova.compute.manager [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c] File /usr/lib/python2.7/site-packages/nova/compute/manager.py, line 4218, in _detach_volume 2014-05-16 16:30:22.331 41419 TRACE nova.compute.manager [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c] encryption=encryption) 2014-05-16 16:30:22.331 41419 TRACE nova.compute.manager [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c] File /usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py, line 1356, in detach_volume 2014-05-16 16:30:22.331 41419 TRACE nova.compute.manager [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c] virt_dom = self._lookup_by_name(instance_name) 2014-05-16 16:30:22.331 41419 TRACE nova.compute.manager [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c] File /usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py, line 3477, in _lookup_by_name 2014-05-16 16:30:22.331 41419 TRACE nova.compute.manager [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c] raise exception.InstanceNotFound(instance_id=instance_name) 2014-05-16 16:30:22.331 41419 TRACE nova.compute.manager [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c] InstanceNotFound: Instance rhel65_113-3e1d0d56-0002 could not be found. 2014-05-16 16:30:22.331 41419 TRACE nova.compute.manager [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c] To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1321082/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1321082] Re: libvirt driver detach_volume fails after migration failure
** Also affects: nova/icehouse Importance: Undecided Status: New ** Changed in: nova/icehouse Status: New = In Progress ** Changed in: nova/icehouse Importance: Undecided = Medium ** Changed in: nova/icehouse Assignee: (unassigned) = Qin Zhao (zhaoqin) -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1321082 Title: libvirt driver detach_volume fails after migration failure Status in OpenStack Compute (Nova): Fix Committed Status in OpenStack Compute (nova) icehouse series: In Progress Bug description: When a VM with an attached iSCSI disk fails to migrate, the rollback methods does not detach the disk from target host. What happens is _lookup_by_name() fails, since the VM does not exist on the target host. In detach_volume(), it is supposed to print a warning based on the correct error code being returned, instead of throwing the exception. However, this is not happening, because _lookup_by_name() throws an InstanceNotFound exception, rather than a libvirt.libvirtError exception. So we also need to catch InstanceNotFound exception, so that detach_volume() can continue to execute as expected. Here's the exception log that I have: 2014-05-16 16:30:22.328 41419 WARNING nova.compute.manager [req-3db28fed-c287-4b41-ac95-9a37a619c75c 0 4be9915c10c8426cbfe948940f7c8af1] [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c] Detaching volume from unknown instance 2014-05-16 16:30:22.331 41419 ERROR nova.compute.manager [req-3db28fed-c287-4b41-ac95-9a37a619c75c 0 4be9915c10c8426cbfe948940f7c8af1] [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c] Failed to detach volume 98a940e5-051f-4d0f-a8c7-859a5079d95e from /dev/vdb 2014-05-16 16:30:22.331 41419 TRACE nova.compute.manager [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c] Traceback (most recent call last): 2014-05-16 16:30:22.331 41419 TRACE nova.compute.manager [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c] File /usr/lib/python2.7/site-packages/nova/compute/manager.py, line 4218, in _detach_volume 2014-05-16 16:30:22.331 41419 TRACE nova.compute.manager [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c] encryption=encryption) 2014-05-16 16:30:22.331 41419 TRACE nova.compute.manager [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c] File /usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py, line 1356, in detach_volume 2014-05-16 16:30:22.331 41419 TRACE nova.compute.manager [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c] virt_dom = self._lookup_by_name(instance_name) 2014-05-16 16:30:22.331 41419 TRACE nova.compute.manager [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c] File /usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py, line 3477, in _lookup_by_name 2014-05-16 16:30:22.331 41419 TRACE nova.compute.manager [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c] raise exception.InstanceNotFound(instance_id=instance_name) 2014-05-16 16:30:22.331 41419 TRACE nova.compute.manager [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c] InstanceNotFound: Instance rhel65_113-3e1d0d56-0002 could not be found. 2014-05-16 16:30:22.331 41419 TRACE nova.compute.manager [instance: 3e1d0d56-3370-4d05-8210-0485fa31757c] To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1321082/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
