Public bug reported: Using a devstack setup, fairly latest!
1) Create a cinder volume (used GlusterFS as the cinder backed) - cv1 2) Attach cv1 to vm1 (vm1 is a nova VM in running state) 3) Create 2 snapshots of vol1 using cinder snapshot-create ... cv1-snap1, cv1-snap2 4) Stop the nova vm vm1 (Note that cinder still reports the volume cv1 as 'in-use') 5) From cinder, delete cv1-snap1. Since cv1-snap1 is _Not_ the active file, nova tries to do a blockCommit and fails with excp below: 2015-07-06 09:33:00.479 ERROR oslo_messaging.rpc.dispatcher [req-695dd8c5-2722-4cf2-ab0a-583b0dacd388 nova service] Exception during message handling: Requested operation is not valid: domain is not running 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher Traceback (most recent call last): 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher executor_callback)) 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher executor_callback) 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 130, in _do_dispatch 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher result = func(ctxt, **new_args) 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 142, in inner 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher return func(*args, **kwargs) 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/exception.py", line 89, in wrapped 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher payload) 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 119, in __exit__ 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/exception.py", line 72, in wrapped 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher return f(self, context, *args, **kw) 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/compute/manager.py", line 2954, in volume_snapshot_delete 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher snapshot_id, delete_info) 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 2024, in volume_snapshot_delete 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher base_file = delete_info['base_file'] 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 119, in __exit__ 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 2017, in volume_snapshot_delete 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher "snapshots.") % ver 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 2003, in _volume_snapshot_delete 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher # paths are maintained relative by qemu. 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 183, in doit 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher result = proxy_call(self._autowrap, f, *args, **kwargs) 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 141, in proxy_call 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher rv = execute(f, *args, **kwargs) 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 122, in execute 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher six.reraise(c, e, tb) 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 80, in tworker 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher rv = meth(*args, **kwargs) 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib64/python2.7/site-packages/libvirt.py", line 642, in blockCommit 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher if ret == -1: raise libvirtError ('virDomainBlockCommit() failed', dom=self) 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher libvirtError: Requested operation is not valid: domain is not running ** Affects: nova Importance: Undecided Status: New ** Tags: blockcommit libvirt nova -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1471726 Title: libvirt: blockCommit fails if domain is not running, for attached cinder volumes Status in OpenStack Compute (Nova): New Bug description: Using a devstack setup, fairly latest! 1) Create a cinder volume (used GlusterFS as the cinder backed) - cv1 2) Attach cv1 to vm1 (vm1 is a nova VM in running state) 3) Create 2 snapshots of vol1 using cinder snapshot-create ... cv1-snap1, cv1-snap2 4) Stop the nova vm vm1 (Note that cinder still reports the volume cv1 as 'in-use') 5) From cinder, delete cv1-snap1. Since cv1-snap1 is _Not_ the active file, nova tries to do a blockCommit and fails with excp below: 2015-07-06 09:33:00.479 ERROR oslo_messaging.rpc.dispatcher [req-695dd8c5-2722-4cf2-ab0a-583b0dacd388 nova service] Exception during message handling: Requested operation is not valid: domain is not running 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher Traceback (most recent call last): 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher executor_callback)) 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher executor_callback) 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 130, in _do_dispatch 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher result = func(ctxt, **new_args) 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 142, in inner 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher return func(*args, **kwargs) 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/exception.py", line 89, in wrapped 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher payload) 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 119, in __exit__ 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/exception.py", line 72, in wrapped 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher return f(self, context, *args, **kw) 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/compute/manager.py", line 2954, in volume_snapshot_delete 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher snapshot_id, delete_info) 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 2024, in volume_snapshot_delete 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher base_file = delete_info['base_file'] 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 119, in __exit__ 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 2017, in volume_snapshot_delete 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher "snapshots.") % ver 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 2003, in _volume_snapshot_delete 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher # paths are maintained relative by qemu. 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 183, in doit 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher result = proxy_call(self._autowrap, f, *args, **kwargs) 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 141, in proxy_call 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher rv = execute(f, *args, **kwargs) 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 122, in execute 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher six.reraise(c, e, tb) 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 80, in tworker 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher rv = meth(*args, **kwargs) 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib64/python2.7/site-packages/libvirt.py", line 642, in blockCommit 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher if ret == -1: raise libvirtError ('virDomainBlockCommit() failed', dom=self) 2015-07-06 09:33:00.479 TRACE oslo_messaging.rpc.dispatcher libvirtError: Requested operation is not valid: domain is not running To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1471726/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp