** Also affects: nova/kilo Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1453666
Title: libvirt: guestfs api makes nova-compute hang Status in OpenStack Compute (nova): Fix Released Status in OpenStack Compute (nova) kilo series: New Bug description: Latest Kilo code. In inspect_capabilities() of nova/virt/disk/vfs/guestfs.py, guestfs api, which is C-extension, will hang nova-compute process when it is invoked. This problem will result in message queue time out error and instance booting failure. And example of this problem is: 2015-05-09 17:07:08.393 4449 DEBUG nova.virt.disk.vfs.api [req-1f7c1104-2679-43a5-bbcb-f73114ce9103 - - - - -] Using primary VFSGuestFS instance_for_image /usr/lib/python2.7/site-packages/nova/virt/disk/vfs/api.py:50 2015-05-09 17:08:35.443 4449 DEBUG nova.virt.disk.vfs.guestfs [req-1f7c1104-2679-43a5-bbcb-f73114ce9103 - - - - -] Setting up appliance for /var/lib/nova/instances/0517e2a9-469c-43f4-a129-f489fc1c8356/disk qcow2 setup /usr/lib/python2.7/site-packages/nova/virt/disk/vfs/guestfs.py:169 2015-05-09 17:08:35.457 4449 DEBUG nova.openstack.common.periodic_task [req-bb78b74b-bed7-450f-bd40-19686aab2c3e - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks /usr/lib/python2.7/site-packages/nova/openstack/common/periodic_task.py:219 2015-05-09 17:08:35.461 4449 INFO oslo_messaging._drivers.impl_rabbit [req-bb78b74b-bed7-450f-bd40-19686aab2c3e - - - - -] Connecting to AMQP server on 127.0.0.1:5671 2015-05-09 17:08:35.472 4449 ERROR nova.compute.manager [-] Instance failed network setup after 1 attempt(s) 2015-05-09 17:08:35.472 4449 TRACE nova.compute.manager Traceback (most recent call last): 2015-05-09 17:08:35.472 4449 TRACE nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1783, in _allocate_network_async 2015-05-09 17:08:35.472 4449 TRACE nova.compute.manager system_metadata=sys_meta) 2015-05-09 17:08:35.472 4449 TRACE nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 739, in _instance_update 2015-05-09 17:08:35.472 4449 TRACE nova.compute.manager **kwargs) 2015-05-09 17:08:35.472 4449 TRACE nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/conductor/api.py", line 308, in instance_update 2015-05-09 17:08:35.472 4449 TRACE nova.compute.manager updates, 'conductor') 2015-05-09 17:08:35.472 4449 TRACE nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/conductor/rpcapi.py", line 194, in instance_update 2015-05-09 17:08:35.472 4449 TRACE nova.compute.manager service=service) 2015-05-09 17:08:35.472 4449 TRACE nova.compute.manager File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/client.py", line 156, in call 2015-05-09 17:08:35.472 4449 TRACE nova.compute.manager retry=self.retry) 2015-05-09 17:08:35.472 4449 TRACE nova.compute.manager File "/usr/lib/python2.7/site-packages/oslo_messaging/transport.py", line 90, in _send 2015-05-09 17:08:35.472 4449 TRACE nova.compute.manager timeout=timeout, retry=retry) 2015-05-09 17:08:35.472 4449 TRACE nova.compute.manager File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 350, in send 2015-05-09 17:08:35.472 4449 TRACE nova.compute.manager retry=retry) 2015-05-09 17:08:35.472 4449 TRACE nova.compute.manager File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 339, in _send 2015-05-09 17:08:35.472 4449 TRACE nova.compute.manager result = self._waiter.wait(msg_id, timeout) 2015-05-09 17:08:35.472 4449 TRACE nova.compute.manager File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 243, in wait 2015-05-09 17:08:35.472 4449 TRACE nova.compute.manager message = self.waiters.get(msg_id, timeout=timeout) 2015-05-09 17:08:35.472 4449 TRACE nova.compute.manager File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 149, in get 2015-05-09 17:08:35.472 4449 TRACE nova.compute.manager 'to message ID %s' % msg_id) 2015-05-09 17:08:35.472 4449 TRACE nova.compute.manager MessagingTimeout: Timed out waiting for a reply to message ID 8ff07520ea8743c997b5017f6638a0df 2015-05-09 17:08:35.472 4449 TRACE nova.compute.manager To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1453666/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp