Public bug reported: In stable branches the eventlet.event.Event.wait() function can raise a eventlet.timeout.Timeout exception when we are waiting for instance event
In below example, a user tried a rebuild of an instance with an invalid image, it caused the instance to enter an error state as expected but the error callback was never called and the RPC reply was never performed as oslo.messaging cannot serialize a eventlet.timeout.Timeout exception This is not an issue in master as the eventlet removal in favor of threading.Event as per https://review.opendev.org/c/openstack/nova/+/949754 We should consider replacing the eventlet.timeout.Timeout exception with InstanceEventTimeout exception same way we do in the above patch that removes eventlet, on stable branches Small reproducer: ----------------- from oslo_messaging._drivers.common import serialize_msg import eventlet.timeout class InstanceEventTimeout(Exception): pass exc1 = InstanceEventTimeout() msg = serialize_msg(exc1) # This works print(msg) exc2 = eventlet.timeout.Timeout() serialize_msg(exc2) # This raises ----- 2025-10-21 10:26:12.311 2297799 INFO nova.compute.manager [None req-f2551701-a80e-4ba2-853a-f84a4e0d76e2 2983a90a6ec14b2eb303d2c8a4b6e57b 345f2ee39b784141bf8642380c365497 - - default default] [instance: 996ea1d6-e537-4e3a-a394-6d9b22984524] Successfully reverted task state from rebuilding on failure for instance. 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server [None req-f2551701-a80e-4ba2-853a-f84a4e0d76e2 2983a90a6ec14b2eb303d2c8a4b6e57b 345f2ee39b784141bf8642380c365497 - - default default] Exception during message handling: ValueError: Cannot convert <Timeout at 0x7f52ad26ed60 seconds=0> to primitive 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/nova/compute/utils.py", line 1453, in decorated_function 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/nova/compute/manager.py", line 203, in decorated_function 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/nova/compute/manager.py", line 3859, in rebuild_instance 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server self._do_rebuild_instance_with_claim( 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/nova/compute/manager.py", line 3945, in _do_rebuild_instance_with_claim 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server self._do_rebuild_instance( 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/nova/compute/manager.py", line 4137, in _do_rebuild_instance 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server self._rebuild_default_impl(**kwargs) 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/nova/compute/manager.py", line 3714, in _rebuild_default_impl 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server self._rebuild_volume_backed_instance( 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/nova/compute/manager.py", line 3637, in _rebuild_volume_backed_instance 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server self.volume_api.reimage_volume( 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib64/python3.9/contextlib.py", line 126, in __exit__ 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server next(self.gen) 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/nova/compute/manager.py", line 559, in wait_for_instance_event 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server self._wait_for_instance_events( 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/nova/compute/manager.py", line 471, in _wait_for_instance_events 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server actual_event = event.wait() 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/nova/compute/manager.py", line 436, in wait 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server instance_event = self.event.wait() 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/eventlet/event.py", line 124, in wait 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server result = hub.switch() 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/eventlet/hubs/hub.py", line 310, in switch 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server return self.greenlet.switch() 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server eventlet.timeout.Timeout: 0 seconds 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server During handling of the above exception, another exception occurred: 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/oslo_messaging/rpc/server.py", line 244, in inner 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server return func(*args, **kwargs) 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/nova/exception_wrapper.py", line 71, in wrapped 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server _emit_versioned_exception_notification( 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 227, in __exit__ 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server self.force_reraise() 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 200, in force_reraise 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server raise self.value 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/nova/exception_wrapper.py", line 63, in wrapped 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server return f(self, context, *args, **kw) 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/nova/compute/manager.py", line 186, in decorated_function 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server LOG.warning("Failed to revert task state for instance. " 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 227, in __exit__ 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server self.force_reraise() 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 200, in force_reraise 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server raise self.value 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/nova/compute/manager.py", line 157, in decorated_function 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/nova/compute/utils.py", line 1453, in decorated_function 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/nova/compute/utils.py", line 1414, in __exit__ 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server objects.InstanceActionEvent.event_finish_with_failure( 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/nova/objects/base.py", line 393, in wrapper 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server return fn.__get__(None, obj)(*args, **kwargs) 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/oslo_versionedobjects/base.py", line 175, in wrapper 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server result = cls.indirection_api.object_class_action_versions( 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/nova/conductor/rpcapi.py", line 240, in object_class_action_versions 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server return cctxt.call(context, 'object_class_action_versions', 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/oslo_messaging/rpc/client.py", line 190, in call 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server result = self.transport._send( 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/oslo_messaging/transport.py", line 123, in _send 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server return self._driver.send(target, ctxt, message, 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 793, in send 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server return self._send(target, ctxt, message, wait_for_reply, timeout, 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 748, in _send 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server msg = rpc_common.serialize_msg(msg) 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/oslo_messaging/_drivers/common.py", line 292, in serialize_msg 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server _MESSAGE_KEY: jsonutils.dumps(raw_msg)} 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/oslo_serialization/jsonutils.py", line 206, in dumps 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server return json.dumps(obj, default=default, **kwargs) 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib64/python3.9/json/__init__.py", line 234, in dumps 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server return cls( 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib64/python3.9/json/encoder.py", line 199, in encode 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server chunks = self.iterencode(o, _one_shot=True) 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib64/python3.9/json/encoder.py", line 257, in iterencode 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server return _iterencode(o, 0) 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/oslo_serialization/jsonutils.py", line 184, in to_primitive 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server raise ValueError("Cannot convert %r to primitive" % (value,)) 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server ValueError: Cannot convert <Timeout at 0x7f52ad26ed60 seconds=0> to primitive ** Affects: nova Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/2129468 Title: [stable-only] eventlet.event.Event.wait() can raise eventlet.timeout.Timeout that cannot be serialized by oslo.messaging Status in OpenStack Compute (nova): New Bug description: In stable branches the eventlet.event.Event.wait() function can raise a eventlet.timeout.Timeout exception when we are waiting for instance event In below example, a user tried a rebuild of an instance with an invalid image, it caused the instance to enter an error state as expected but the error callback was never called and the RPC reply was never performed as oslo.messaging cannot serialize a eventlet.timeout.Timeout exception This is not an issue in master as the eventlet removal in favor of threading.Event as per https://review.opendev.org/c/openstack/nova/+/949754 We should consider replacing the eventlet.timeout.Timeout exception with InstanceEventTimeout exception same way we do in the above patch that removes eventlet, on stable branches Small reproducer: ----------------- from oslo_messaging._drivers.common import serialize_msg import eventlet.timeout class InstanceEventTimeout(Exception): pass exc1 = InstanceEventTimeout() msg = serialize_msg(exc1) # This works print(msg) exc2 = eventlet.timeout.Timeout() serialize_msg(exc2) # This raises ----- 2025-10-21 10:26:12.311 2297799 INFO nova.compute.manager [None req-f2551701-a80e-4ba2-853a-f84a4e0d76e2 2983a90a6ec14b2eb303d2c8a4b6e57b 345f2ee39b784141bf8642380c365497 - - default default] [instance: 996ea1d6-e537-4e3a-a394-6d9b22984524] Successfully reverted task state from rebuilding on failure for instance. 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server [None req-f2551701-a80e-4ba2-853a-f84a4e0d76e2 2983a90a6ec14b2eb303d2c8a4b6e57b 345f2ee39b784141bf8642380c365497 - - default default] Exception during message handling: ValueError: Cannot convert <Timeout at 0x7f52ad26ed60 seconds=0> to primitive 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/nova/compute/utils.py", line 1453, in decorated_function 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/nova/compute/manager.py", line 203, in decorated_function 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/nova/compute/manager.py", line 3859, in rebuild_instance 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server self._do_rebuild_instance_with_claim( 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/nova/compute/manager.py", line 3945, in _do_rebuild_instance_with_claim 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server self._do_rebuild_instance( 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/nova/compute/manager.py", line 4137, in _do_rebuild_instance 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server self._rebuild_default_impl(**kwargs) 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/nova/compute/manager.py", line 3714, in _rebuild_default_impl 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server self._rebuild_volume_backed_instance( 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/nova/compute/manager.py", line 3637, in _rebuild_volume_backed_instance 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server self.volume_api.reimage_volume( 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib64/python3.9/contextlib.py", line 126, in __exit__ 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server next(self.gen) 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/nova/compute/manager.py", line 559, in wait_for_instance_event 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server self._wait_for_instance_events( 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/nova/compute/manager.py", line 471, in _wait_for_instance_events 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server actual_event = event.wait() 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/nova/compute/manager.py", line 436, in wait 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server instance_event = self.event.wait() 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/eventlet/event.py", line 124, in wait 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server result = hub.switch() 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/eventlet/hubs/hub.py", line 310, in switch 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server return self.greenlet.switch() 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server eventlet.timeout.Timeout: 0 seconds 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server During handling of the above exception, another exception occurred: 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/oslo_messaging/rpc/server.py", line 244, in inner 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server return func(*args, **kwargs) 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/nova/exception_wrapper.py", line 71, in wrapped 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server _emit_versioned_exception_notification( 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 227, in __exit__ 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server self.force_reraise() 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 200, in force_reraise 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server raise self.value 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/nova/exception_wrapper.py", line 63, in wrapped 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server return f(self, context, *args, **kw) 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/nova/compute/manager.py", line 186, in decorated_function 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server LOG.warning("Failed to revert task state for instance. " 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 227, in __exit__ 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server self.force_reraise() 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 200, in force_reraise 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server raise self.value 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/nova/compute/manager.py", line 157, in decorated_function 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/nova/compute/utils.py", line 1453, in decorated_function 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/nova/compute/utils.py", line 1414, in __exit__ 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server objects.InstanceActionEvent.event_finish_with_failure( 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/nova/objects/base.py", line 393, in wrapper 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server return fn.__get__(None, obj)(*args, **kwargs) 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/oslo_versionedobjects/base.py", line 175, in wrapper 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server result = cls.indirection_api.object_class_action_versions( 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/nova/conductor/rpcapi.py", line 240, in object_class_action_versions 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server return cctxt.call(context, 'object_class_action_versions', 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/oslo_messaging/rpc/client.py", line 190, in call 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server result = self.transport._send( 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/oslo_messaging/transport.py", line 123, in _send 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server return self._driver.send(target, ctxt, message, 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 793, in send 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server return self._send(target, ctxt, message, wait_for_reply, timeout, 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 748, in _send 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server msg = rpc_common.serialize_msg(msg) 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/oslo_messaging/_drivers/common.py", line 292, in serialize_msg 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server _MESSAGE_KEY: jsonutils.dumps(raw_msg)} 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/oslo_serialization/jsonutils.py", line 206, in dumps 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server return json.dumps(obj, default=default, **kwargs) 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib64/python3.9/json/__init__.py", line 234, in dumps 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server return cls( 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib64/python3.9/json/encoder.py", line 199, in encode 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server chunks = self.iterencode(o, _one_shot=True) 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib64/python3.9/json/encoder.py", line 257, in iterencode 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server return _iterencode(o, 0) 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.9/site-packages/oslo_serialization/jsonutils.py", line 184, in to_primitive 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server raise ValueError("Cannot convert %r to primitive" % (value,)) 2025-10-21 10:26:12.322 2297799 ERROR oslo_messaging.rpc.server ValueError: Cannot convert <Timeout at 0x7f52ad26ed60 seconds=0> to primitive To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/2129468/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
