Public bug reported:
++++++++++++
Env Details:
++++++++++++
OSA: 2024.1 <Caracal>
OS: Ubuntu 22.04
* We have a Offline deployed openstack enviroenmmtn, The environemnt is up and
running, Couple of test cases we performed,
* able to create instance from image and volume, evacuation, migration
everything is working.
* We saw one patern whenever we are running ``opensatck hypervisor list
`` , sometime its working sometime its giving an exception, out of 5
times only 2/3 times its working
```
RESP BODY: {"computeFault": {"code": 500, "message": "Unexpected API Error.
Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API
log if possible.\n<class 'oslo_messaging.exceptions.MessagingTimeout'>"}}
GET call to compute for http://192.168.131.200:8774/v2.1/os-hypervisors/detail
used request id req-ad28b6c6-3277-4e94-9023-0478ac71f33a
HttpException: 500: Server Error for url:
http://192.168.131.200:8774/v2.1/os-hypervisors/detail, Unexpected API Error.
Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API
log if possible.
<class 'oslo_messaging.exceptions.MessagingTimeout'>
```
```
Jul 30 08:33:57 offline20241.ct.lan nova-api-wsgi[2061714]: 2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi [None
req-12ea9d0c-f983-468b-afa3-cb6562be492e cfc72fc0bc3d4ba09ef9756ea2fb6395
2f5a2a06638942cbaaeeb466b2e17e10 - - default default] Unexpected exception in
API method: oslo_messaging.exceptions.MessagingTimeout: Timed out waiting for a
reply to message ID a48c30c8be8d4765a6bbf58b6916ec0f
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi Traceback (most recent call
last):
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi File
"/openstack/venvs/nova-29.2.3.dev1/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py",
line 511, in get
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi return
self._queues[msg_id].get(block=True, timeout=timeout)
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi File
"/openstack/venvs/nova-29.2.3.dev1/lib/python3.10/site-packages/eventlet/queue.py",
line 320, in get
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi return waiter.wait()
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi File
"/openstack/venvs/nova-29.2.3.dev1/lib/python3.10/site-packages/eventlet/queue.py",
line 139, in wait
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi return get_hub().switch()
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi File
"/openstack/venvs/nova-29.2.3.dev1/lib/python3.10/site-packages/eventlet/hubs/hub.py",
line 310, in switch
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi return
self.greenlet.switch()
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi _queue.Empty
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi During handling of the above
exception, another exception occurred:
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi Traceback (most recent call
last):
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi File
"/openstack/venvs/nova-29.2.3.dev1/lib/python3.10/site-packages/nova/api/openstack/wsgi.py",
line 658, in wrapped
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi return f(*args, **kwargs)
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi File
"/openstack/venvs/nova-29.2.3.dev1/lib/python3.10/site-packages/nova/api/openstack/compute/hypervisors.py",
line 273, in detail
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi return self._detail(req,
limit=limit, marker=marker, links=True)
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi File
"/openstack/venvs/nova-29.2.3.dev1/lib/python3.10/site-packages/nova/api/openstack/compute/hypervisors.py",
line 290, in _detail
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi return
self._get_hypervisors(req, detail=True, limit=limit,
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi File
"/openstack/venvs/nova-29.2.3.dev1/lib/python3.10/site-packages/nova/api/openstack/compute/hypervisors.py",
line 215, in _get_hypervisors
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi hypervisor =
self._view_hypervisor(
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi File
"/openstack/venvs/nova-29.2.3.dev1/lib/python3.10/site-packages/nova/api/openstack/compute/hypervisors.py",
line 106, in _view_hypervisor
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi hyp_dict['uptime'] =
self.host_api.get_host_uptime(
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi File
"/openstack/venvs/nova-29.2.3.dev1/lib/python3.10/site-packages/nova/compute/api.py",
line 6042, in targeted
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi return fn(self, context,
host, *args, **kwargs)
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi File
"/openstack/venvs/nova-29.2.3.dev1/lib/python3.10/site-packages/nova/compute/api.py",
line 6157, in get_host_uptime
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi return
self.rpcapi.get_host_uptime(context, host=host_name)
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi File
"/openstack/venvs/nova-29.2.3.dev1/lib/python3.10/site-packages/nova/compute/rpcapi.py",
line 1311, in get_host_uptime
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi return cctxt.call(ctxt,
'get_host_uptime')
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi File
"/openstack/venvs/nova-29.2.3.dev1/lib/python3.10/site-packages/oslo_messaging/rpc/client.py",
line 190, in call
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi result =
self.transport._send(
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi File
"/openstack/venvs/nova-29.2.3.dev1/lib/python3.10/site-packages/oslo_messaging/transport.py",
line 123, in _send
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi return
self._driver.send(target, ctxt, message,
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi File
"/openstack/venvs/nova-29.2.3.dev1/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py",
line 793, in send
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi return
self._send(target, ctxt, message, wait_for_reply, timeout,
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi File
"/openstack/venvs/nova-29.2.3.dev1/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py",
line 782, in _send
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi result =
self._waiter.wait(msg_id, timeout,
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi File
"/openstack/venvs/nova-29.2.3.dev1/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py",
line 647, in wait
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi message =
self.waiters.get(msg_id, timeout=timeout)
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi File
"/openstack/venvs/nova-29.2.3.dev1/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py",
line 513, in get
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi raise
oslo_messaging.MessagingTimeout(
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi
oslo_messaging.exceptions.MessagingTimeout: Timed out waiting for a reply to
message ID a48c30c8be8d4765a6bbf58b6916ec0f
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi
Jul 30 08:33:57 offline20241.ct.lan nova-api-wsgi[2061714]: 2025-07-30
08:33:57.521 2061714 INFO nova.api.openstack.wsgi [None
req-12ea9d0c-f983-468b-afa3-cb6562be492e cfc72fc0bc3d4ba09ef9756ea2fb6395
2f5a2a06638942cbaaeeb466b2e17e10 - - default default] HTTP exception thrown:
Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and
attach the Nova API log if possible.
```
** Affects: nova
Importance: Undecided
Status: New
** Tags: nova openstack-ansible
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/2119106
Title:
[Caracal][Nova][OFFLINE] - API method:
oslo_messaging.exceptions.MessagingTimeout: Timed out waiting for a
reply to message ID
Status in OpenStack Compute (nova):
New
Bug description:
++++++++++++
Env Details:
++++++++++++
OSA: 2024.1 <Caracal>
OS: Ubuntu 22.04
* We have a Offline deployed openstack enviroenmmtn, The environemnt is up
and running, Couple of test cases we performed,
* able to create instance from image and volume, evacuation, migration
everything is working.
* We saw one patern whenever we are running ``opensatck hypervisor
list `` , sometime its working sometime its giving an exception, out
of 5 times only 2/3 times its working
```
RESP BODY: {"computeFault": {"code": 500, "message": "Unexpected API Error.
Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API
log if possible.\n<class 'oslo_messaging.exceptions.MessagingTimeout'>"}}
GET call to compute for
http://192.168.131.200:8774/v2.1/os-hypervisors/detail used request id
req-ad28b6c6-3277-4e94-9023-0478ac71f33a
HttpException: 500: Server Error for url:
http://192.168.131.200:8774/v2.1/os-hypervisors/detail, Unexpected API Error.
Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API
log if possible.
<class 'oslo_messaging.exceptions.MessagingTimeout'>
```
```
Jul 30 08:33:57 offline20241.ct.lan nova-api-wsgi[2061714]: 2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi [None
req-12ea9d0c-f983-468b-afa3-cb6562be492e cfc72fc0bc3d4ba09ef9756ea2fb6395
2f5a2a06638942cbaaeeb466b2e17e10 - - default default] Unexpected exception in
API method: oslo_messaging.exceptions.MessagingTimeout: Timed out waiting for a
reply to message ID a48c30c8be8d4765a6bbf58b6916ec0f
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi Traceback (most recent call
last):
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi File
"/openstack/venvs/nova-29.2.3.dev1/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py",
line 511, in get
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi return
self._queues[msg_id].get(block=True, timeout=timeout)
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi File
"/openstack/venvs/nova-29.2.3.dev1/lib/python3.10/site-packages/eventlet/queue.py",
line 320, in get
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi return waiter.wait()
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi File
"/openstack/venvs/nova-29.2.3.dev1/lib/python3.10/site-packages/eventlet/queue.py",
line 139, in wait
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi return get_hub().switch()
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi File
"/openstack/venvs/nova-29.2.3.dev1/lib/python3.10/site-packages/eventlet/hubs/hub.py",
line 310, in switch
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi return
self.greenlet.switch()
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi _queue.Empty
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi During handling of the above
exception, another exception occurred:
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi Traceback (most recent call
last):
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi File
"/openstack/venvs/nova-29.2.3.dev1/lib/python3.10/site-packages/nova/api/openstack/wsgi.py",
line 658, in wrapped
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi return f(*args, **kwargs)
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi File
"/openstack/venvs/nova-29.2.3.dev1/lib/python3.10/site-packages/nova/api/openstack/compute/hypervisors.py",
line 273, in detail
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi return self._detail(req,
limit=limit, marker=marker, links=True)
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi File
"/openstack/venvs/nova-29.2.3.dev1/lib/python3.10/site-packages/nova/api/openstack/compute/hypervisors.py",
line 290, in _detail
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi return
self._get_hypervisors(req, detail=True, limit=limit,
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi File
"/openstack/venvs/nova-29.2.3.dev1/lib/python3.10/site-packages/nova/api/openstack/compute/hypervisors.py",
line 215, in _get_hypervisors
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi hypervisor =
self._view_hypervisor(
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi File
"/openstack/venvs/nova-29.2.3.dev1/lib/python3.10/site-packages/nova/api/openstack/compute/hypervisors.py",
line 106, in _view_hypervisor
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi hyp_dict['uptime'] =
self.host_api.get_host_uptime(
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi File
"/openstack/venvs/nova-29.2.3.dev1/lib/python3.10/site-packages/nova/compute/api.py",
line 6042, in targeted
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi return fn(self, context,
host, *args, **kwargs)
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi File
"/openstack/venvs/nova-29.2.3.dev1/lib/python3.10/site-packages/nova/compute/api.py",
line 6157, in get_host_uptime
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi return
self.rpcapi.get_host_uptime(context, host=host_name)
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi File
"/openstack/venvs/nova-29.2.3.dev1/lib/python3.10/site-packages/nova/compute/rpcapi.py",
line 1311, in get_host_uptime
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi return cctxt.call(ctxt,
'get_host_uptime')
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi File
"/openstack/venvs/nova-29.2.3.dev1/lib/python3.10/site-packages/oslo_messaging/rpc/client.py",
line 190, in call
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi result =
self.transport._send(
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi File
"/openstack/venvs/nova-29.2.3.dev1/lib/python3.10/site-packages/oslo_messaging/transport.py",
line 123, in _send
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi return
self._driver.send(target, ctxt, message,
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi File
"/openstack/venvs/nova-29.2.3.dev1/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py",
line 793, in send
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi return
self._send(target, ctxt, message, wait_for_reply, timeout,
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi File
"/openstack/venvs/nova-29.2.3.dev1/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py",
line 782, in _send
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi result =
self._waiter.wait(msg_id, timeout,
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi File
"/openstack/venvs/nova-29.2.3.dev1/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py",
line 647, in wait
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi message =
self.waiters.get(msg_id, timeout=timeout)
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi File
"/openstack/venvs/nova-29.2.3.dev1/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py",
line 513, in get
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi raise
oslo_messaging.MessagingTimeout(
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi
oslo_messaging.exceptions.MessagingTimeout: Timed out waiting for a reply to
message ID a48c30c8be8d4765a6bbf58b6916ec0f
2025-07-30
08:33:57.518 2061714 ERROR nova.api.openstack.wsgi
Jul 30 08:33:57 offline20241.ct.lan nova-api-wsgi[2061714]: 2025-07-30
08:33:57.521 2061714 INFO nova.api.openstack.wsgi [None
req-12ea9d0c-f983-468b-afa3-cb6562be492e cfc72fc0bc3d4ba09ef9756ea2fb6395
2f5a2a06638942cbaaeeb466b2e17e10 - - default default] HTTP exception thrown:
Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and
attach the Nova API log if possible.
```
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/2119106/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : [email protected]
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
