Public bug reported:
Nova sometimes raises an exception when running the periodic instance deleted, see below. The error is quite spurious, and it is not happening always so I cannot get the reason why this is happening. The periodic reclaim is trying to delete the instance, but the DB is not returning the instance as it is marked as deleted. ERROR nova.compute.manager Traceback (most recent call last): ERROR nova.compute.manager ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/conductor/manager.py", line 447, in _object_dispatch ERROR nova.compute.manager return getattr(target, method)(*args, **kwargs) ERROR nova.compute.manager ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 213, in wrapper ERROR nova.compute.manager return fn(self, *args, **kwargs) ERROR nova.compute.manager ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/objects/instance.py", line 673, in save ERROR nova.compute.manager columns_to_join=_expected_cols(expected_attrs)) ERROR nova.compute.manager ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/db/api.py", line 765, in instance_update_and_get_original ERROR nova.compute.manager expected=expected) ERROR nova.compute.manager ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/api.py", line 216, in wrapper ERROR nova.compute.manager return f(*args, **kwargs) ERROR nova.compute.manager ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/oslo_db/api.py", line 146, in wrapper ERROR nova.compute.manager ectxt.value = e.inner_exc ERROR nova.compute.manager ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 195, in __exit__ ERROR nova.compute.manager six.reraise(self.type_, self.value, self.tb) ERROR nova.compute.manager ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/oslo_db/api.py", line 136, in wrapper ERROR nova.compute.manager return f(*args, **kwargs) ERROR nova.compute.manager ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/api.py", line 2478, in instance_update_and_get_original ERROR nova.compute.manager session=session) ERROR nova.compute.manager ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/api.py", line 1733, in _instance_get_by_uuid ERROR nova.compute.manager raise exception.InstanceNotFound(instance_id=uuid) ERROR nova.compute.manager ERROR nova.compute.manager InstanceNotFound: Instance aceb4514-915e-4687-ac99-20f6b7d5ed6d could not be found. ERROR nova.compute.manager ERROR nova.compute.manager WARNING nova.compute.manager [req-e62c7e7c-0799-4f87-8f06-a46cf5663733 - - - - -] [instance: aceb4514-915e-4687-ac99-20f6b7d5ed6d] Periodic reclaim failed to delete instance: Instance aceb4514-915e-4687-ac99-20f6b7d5ed6d could not be found. As I said, this is an spurious error, but from time to time we get a bunch of these errors and we need to manually delete those instances. ** Affects: nova Importance: Undecided Assignee: Alvaro Lopez (aloga) Status: New ** Changed in: nova Assignee: (unassigned) => Alvaro Lopez (aloga) -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1587356 Title: nova raises "InstanceNotFound" when running the periodic reclaim Status in OpenStack Compute (nova): New Bug description: Nova sometimes raises an exception when running the periodic instance deleted, see below. The error is quite spurious, and it is not happening always so I cannot get the reason why this is happening. The periodic reclaim is trying to delete the instance, but the DB is not returning the instance as it is marked as deleted. ERROR nova.compute.manager Traceback (most recent call last): ERROR nova.compute.manager ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/conductor/manager.py", line 447, in _object_dispatch ERROR nova.compute.manager return getattr(target, method)(*args, **kwargs) ERROR nova.compute.manager ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 213, in wrapper ERROR nova.compute.manager return fn(self, *args, **kwargs) ERROR nova.compute.manager ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/objects/instance.py", line 673, in save ERROR nova.compute.manager columns_to_join=_expected_cols(expected_attrs)) ERROR nova.compute.manager ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/db/api.py", line 765, in instance_update_and_get_original ERROR nova.compute.manager expected=expected) ERROR nova.compute.manager ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/api.py", line 216, in wrapper ERROR nova.compute.manager return f(*args, **kwargs) ERROR nova.compute.manager ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/oslo_db/api.py", line 146, in wrapper ERROR nova.compute.manager ectxt.value = e.inner_exc ERROR nova.compute.manager ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 195, in __exit__ ERROR nova.compute.manager six.reraise(self.type_, self.value, self.tb) ERROR nova.compute.manager ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/oslo_db/api.py", line 136, in wrapper ERROR nova.compute.manager return f(*args, **kwargs) ERROR nova.compute.manager ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/api.py", line 2478, in instance_update_and_get_original ERROR nova.compute.manager session=session) ERROR nova.compute.manager ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/api.py", line 1733, in _instance_get_by_uuid ERROR nova.compute.manager raise exception.InstanceNotFound(instance_id=uuid) ERROR nova.compute.manager ERROR nova.compute.manager InstanceNotFound: Instance aceb4514-915e-4687-ac99-20f6b7d5ed6d could not be found. ERROR nova.compute.manager ERROR nova.compute.manager WARNING nova.compute.manager [req-e62c7e7c-0799-4f87-8f06-a46cf5663733 - - - - -] [instance: aceb4514-915e-4687-ac99-20f6b7d5ed6d] Periodic reclaim failed to delete instance: Instance aceb4514-915e-4687-ac99-20f6b7d5ed6d could not be found. As I said, this is an spurious error, but from time to time we get a bunch of these errors and we need to manually delete those instances. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1587356/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp