** Also affects: libvirt (Ubuntu)
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1673483
Title:
libvirt: test_attach_volume_shelved_or_offload_server times out
waiting for device detach (which fails)
Status in OpenStack Compute (nova):
Confirmed
Status in libvirt package in Ubuntu:
New
Bug description:
http://logs.openstack.org/75/446175/1/gate/gate-tempest-dsvm-neutron-
full-ubuntu-xenial/479c7bf/console.html#_2017-03-16_01_31_33_688563
The test teardown times out waiting for the volume to be detached from
the server because the volume detach fails due to a libvirt i/o error
during detach:
http://logs.openstack.org/75/446175/1/gate/gate-tempest-dsvm-neutron-
full-ubuntu-
xenial/479c7bf/logs/screen-n-cpu.txt.gz?#_2017-03-16_01_01_32_897
2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager
[req-ac93339e-9eb1-4104-8743-5e5626253ab8
tempest-AttachVolumeShelveTestJSON-116820638
tempest-AttachVolumeShelveTestJSON-116820638] [instance:
5f67c47b-f7d7-4063-88f1-8f7b6177fcb9] Failed to detach volume
dc49318a-e51f-4812-97a3-64bf93e522eb from /dev/vdc
2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [instance:
5f67c47b-f7d7-4063-88f1-8f7b6177fcb9] Traceback (most recent call last):
2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [instance:
5f67c47b-f7d7-4063-88f1-8f7b6177fcb9] File
"/opt/stack/new/nova/nova/compute/manager.py", line 4833, in
_driver_detach_volume
2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [instance:
5f67c47b-f7d7-4063-88f1-8f7b6177fcb9] encryption=encryption)
2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [instance:
5f67c47b-f7d7-4063-88f1-8f7b6177fcb9] File
"/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 1334, in detach_volume
2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [instance:
5f67c47b-f7d7-4063-88f1-8f7b6177fcb9] live=live)
2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [instance:
5f67c47b-f7d7-4063-88f1-8f7b6177fcb9] File
"/opt/stack/new/nova/nova/virt/libvirt/guest.py", line 414, in
detach_device_with_retry
2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [instance:
5f67c47b-f7d7-4063-88f1-8f7b6177fcb9] _try_detach_device(conf, persistent,
live)
2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [instance:
5f67c47b-f7d7-4063-88f1-8f7b6177fcb9] File
"/opt/stack/new/nova/nova/virt/libvirt/guest.py", line 408, in
_try_detach_device
2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [instance:
5f67c47b-f7d7-4063-88f1-8f7b6177fcb9] raise
exception.DeviceNotFound(device=device)
2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [instance:
5f67c47b-f7d7-4063-88f1-8f7b6177fcb9] File
"/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in
__exit__
2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [instance:
5f67c47b-f7d7-4063-88f1-8f7b6177fcb9] self.force_reraise()
2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [instance:
5f67c47b-f7d7-4063-88f1-8f7b6177fcb9] File
"/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in
force_reraise
2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [instance:
5f67c47b-f7d7-4063-88f1-8f7b6177fcb9] six.reraise(self.type_, self.value,
self.tb)
2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [instance:
5f67c47b-f7d7-4063-88f1-8f7b6177fcb9] File
"/opt/stack/new/nova/nova/virt/libvirt/guest.py", line 393, in
_try_detach_device
2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [instance:
5f67c47b-f7d7-4063-88f1-8f7b6177fcb9] self.detach_device(conf,
persistent=persistent, live=live)
2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [instance:
5f67c47b-f7d7-4063-88f1-8f7b6177fcb9] File
"/opt/stack/new/nova/nova/virt/libvirt/guest.py", line 450, in detach_device
2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [instance:
5f67c47b-f7d7-4063-88f1-8f7b6177fcb9]
self._domain.detachDeviceFlags(device_xml, flags=flags)
2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [instance:
5f67c47b-f7d7-4063-88f1-8f7b6177fcb9] File
"/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 186, in doit
2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [instance:
5f67c47b-f7d7-4063-88f1-8f7b6177fcb9] result = proxy_call(self._autowrap,
f, *args, **kwargs)
2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [instance:
5f67c47b-f7d7-4063-88f1-8f7b6177fcb9] File
"/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 144, in
proxy_call
2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [instance:
5f67c47b-f7d7-4063-88f1-8f7b6177fcb9] rv = execute(f, *args, **kwargs)
2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [instance:
5f67c47b-f7d7-4063-88f1-8f7b6177fcb9] File
"/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 125, in execute
2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [instance:
5f67c47b-f7d7-4063-88f1-8f7b6177fcb9] six.reraise(c, e, tb)
2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [instance:
5f67c47b-f7d7-4063-88f1-8f7b6177fcb9] File
"/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 83, in tworker
2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [instance:
5f67c47b-f7d7-4063-88f1-8f7b6177fcb9] rv = meth(*args, **kwargs)
2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [instance:
5f67c47b-f7d7-4063-88f1-8f7b6177fcb9] File
"/usr/local/lib/python2.7/dist-packages/libvirt.py", line 1192, in
detachDeviceFlags
2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [instance:
5f67c47b-f7d7-4063-88f1-8f7b6177fcb9] if ret == -1: raise libvirtError
('virDomainDetachDeviceFlags() failed', dom=self)
2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [instance:
5f67c47b-f7d7-4063-88f1-8f7b6177fcb9] libvirtError: End of file while reading
data: Input/output error
2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [instance:
5f67c47b-f7d7-4063-88f1-8f7b6177fcb9]
It's not clear if we're trying the detach twice, and the first time
it's OK and then the 2nd fails with the i/o error, which is not in the
list of expected libvirt errors we handle in this case.
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1673483/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
