Given the clearly invalid python in the stack trace that mriedem pointed out, marking as Invalid.
** Changed in: nova Status: Incomplete => Invalid ** Changed in: nova Importance: High => Low -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1535346 Title: Direct Snapshot in Ceph patch causes nova live-migration to break Status in OpenStack Compute (nova): Invalid Bug description: Greetings, When trying to use nova live-migration we do have this trace on the compute node logs (nova-compute.log) The Vm stays in "MIGRATING" state, the VM is still up on the originating node, and never moves. VERSION IN USE: ( KILO v1 ) root@compute2:~# dpkg -l | grep nova ii nova-common 1:2015.1.0-0ubuntu1~cloud0 all OpenStack Compute - common files ii nova-compute 1:2015.1.0-0ubuntu1~cloud0 all OpenStack Compute - compute node base ii nova-compute-kvm 1:2015.1.0-0ubuntu1~cloud0 all OpenStack Compute - compute node (KVM) ii nova-compute-libvirt 1:2015.1.0-0ubuntu1~cloud0 all OpenStack Compute - compute node libvirt support ii python-nova 1:2015.1.0-0ubuntu1~cloud0 all OpenStack Compute Python libraries ii python-novaclient 1:2.22.0-0ubuntu1~cloud0 all client library for OpenStack Compute API CMD: nova migrate-live 6b91ffb3-bf84-48e9-b07d-99603c89055c compute4 LOGS (/var/log/nova/nova-compute.log): 2016-01-18 09:55:46.160 18571 ERROR oslo_messaging.rpc.dispatcher [req-fae37d19-0867-4796-9df3-b08be2120d15 ad8ac2392f41482e887c6b44402a641d dfcd0863f170402689d89771da0ea3ff - - -] Exception during message handling: 'module' object has no attribute 'spawn' 2016-01-18 09:55:46.160 18571 TRACE oslo_messaging.rpc.dispatcher Traceback (most recent call last): 2016-01-18 09:55:46.160 18571 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply 2016-01-18 09:55:46.160 18571 TRACE oslo_messaging.rpc.dispatcher executor_callback)) 2016-01-18 09:55:46.160 18571 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch 2016-01-18 09:55:46.160 18571 TRACE oslo_messaging.rpc.dispatcher executor_callback) 2016-01-18 09:55:46.160 18571 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 130, in _do_dispatch 2016-01-18 09:55:46.160 18571 TRACE oslo_messaging.rpc.dispatcher result = func(ctxt, **new_args) 2016-01-18 09:55:46.160 18571 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 6668, in live_migration 2016-01-18 09:55:46.160 18571 TRACE oslo_messaging.rpc.dispatcher migrate_data=migrate_data) 2016-01-18 09:55:46.160 18571 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 88, in wrapped 2016-01-18 09:55:46.160 18571 TRACE oslo_messaging.rpc.dispatcher payload) 2016-01-18 09:55:46.160 18571 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 85, in __exit__ 2016-01-18 09:55:46.160 18571 TRACE oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) 2016-01-18 09:55:46.160 18571 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 71, in wrapped 2016-01-18 09:55:46.160 18571 TRACE oslo_messaging.rpc.dispatcher return f(self, context, *args, **kw) 2016-01-18 09:55:46.160 18571 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 355, in decorated_function 2016-01-18 09:55:46.160 18571 TRACE oslo_messaging.rpc.dispatcher kwargs['instance'], e, sys.exc_info()) 2016-01-18 09:55:46.160 18571 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 85, in __exit__ 2016-01-18 09:55:46.160 18571 TRACE oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) 2016-01-18 09:55:46.160 18571 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 343, in decorated_function 2016-01-18 09:55:46.160 18571 TRACE oslo_messaging.rpc.dispatcher return function(self, context, *args, **kwargs) 2016-01-18 09:55:46.160 18571 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 5237, in live_migration 2016-01-18 09:55:46.160 18571 TRACE oslo_messaging.rpc.dispatcher block_migration, migrate_data) 2016-01-18 09:55:46.160 18571 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 5330, in live_migration 2016-01-18 09:55:46.160 18571 TRACE oslo_messaging.rpc.dispatcher utils.spawn(self._live_migration, context, instance, dest, 2016-01-18 09:55:46.160 18571 TRACE oslo_messaging.rpc.dispatcher AttributeError: 'module' object has no attribute 'spawn' 2016-01-18 09:55:46.160 18571 TRACE oslo_messaging.rpc.dispatcher CMD: nova host-evacuate-live compute8 LOGS (/var/log/nova/nova-compute.log): 2016-01-18 10:17:09.807 14122 ERROR oslo_messaging.rpc.dispatcher [req-53f79708-4c7c-4405-a116-ed605fc255a4 ad8ac2392f41482e887c6b44402a641d dfcd0863f170402689d89771da0ea3ff - - -] Exception during message handling: 'module' object has no attribute 'spawn' 2016-01-18 10:17:09.807 14122 TRACE oslo_messaging.rpc.dispatcher Traceback (most recent call last): 2016-01-18 10:17:09.807 14122 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply 2016-01-18 10:17:09.807 14122 TRACE oslo_messaging.rpc.dispatcher executor_callback)) 2016-01-18 10:17:09.807 14122 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch 2016-01-18 10:17:09.807 14122 TRACE oslo_messaging.rpc.dispatcher executor_callback) 2016-01-18 10:17:09.807 14122 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 130, in _do_dispatch 2016-01-18 10:17:09.807 14122 TRACE oslo_messaging.rpc.dispatcher result = func(ctxt, **new_args) 2016-01-18 10:17:09.807 14122 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 6668, in live_migration 2016-01-18 10:17:09.807 14122 TRACE oslo_messaging.rpc.dispatcher migrate_data=migrate_data) 2016-01-18 10:17:09.807 14122 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 88, in wrapped 2016-01-18 10:17:09.807 14122 TRACE oslo_messaging.rpc.dispatcher payload) 2016-01-18 10:17:09.807 14122 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 85, in __exit__ 2016-01-18 10:17:09.807 14122 TRACE oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) 2016-01-18 10:17:09.807 14122 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 71, in wrapped 2016-01-18 10:17:09.807 14122 TRACE oslo_messaging.rpc.dispatcher return f(self, context, *args, **kw) 2016-01-18 10:17:09.807 14122 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 355, in decorated_function 2016-01-18 10:17:09.807 14122 TRACE oslo_messaging.rpc.dispatcher kwargs['instance'], e, sys.exc_info()) 2016-01-18 10:17:09.807 14122 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 85, in __exit__ 2016-01-18 10:17:09.807 14122 TRACE oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) 2016-01-18 10:17:09.807 14122 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 343, in decorated_function 2016-01-18 10:17:09.807 14122 TRACE oslo_messaging.rpc.dispatcher return function(self, context, *args, **kwargs) 2016-01-18 10:17:09.807 14122 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 5237, in live_migration 2016-01-18 10:17:09.807 14122 TRACE oslo_messaging.rpc.dispatcher block_migration, migrate_data) 2016-01-18 10:17:09.807 14122 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 5330, in live_migration 2016-01-18 10:17:09.807 14122 TRACE oslo_messaging.rpc.dispatcher utils.spawn(self._live_migration, context, instance, dest, 2016-01-18 10:17:09.807 14122 TRACE oslo_messaging.rpc.dispatcher AttributeError: 'module' object has no attribute 'spawn' 2016-01-18 10:17:09.807 14122 TRACE oslo_messaging.rpc.dispatcher Is there something we did wrong ? [VM is still up and running, only state stuck in MIGRATING; had to nova reset-state --active vMs.] Thanks in advance for your help. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1535346/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp