Reviewed: https://review.openstack.org/565183 Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=1a0179f7ab9b9c7f07ce9c4dd681486de5a4443a Submitter: Zuul Branch: master
commit 1a0179f7ab9b9c7f07ce9c4dd681486de5a4443a Author: Radoslav Gerganov <rgerga...@vmware.com> Date: Mon Apr 30 13:18:26 2018 +0300 Handle PortNotFoundClient exception when getting ports There could be a race between deleting an instance and retrieving its port groups from Neutron. In this case PortNotFoundClient is raised and it can be safely ignored. Change-Id: I31c9ea8628c6f3985f8e9118d9687bbfb8789b68 Closes-Bug: #1767058 ** Changed in: nova Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1767058 Title: Rally test boot_and_delete_server_with_secgroups fails Status in OpenStack Compute (nova): Fix Released Bug description: When running rally scenarios the test fails with PortNotFoundClient exception. From the logs we see the following: Failure: ------- ▼ 11 GetResourceFailure Failed to get the resource <Server: s_rally_41596539_fMbobPdS>: The server has either erred or is incapable of performing the requested operation. (HTTP 500) (Request-ID: req-a9471015-e72e-42cf-acc3-8b361415d91e) Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/rally/task/runner.py", line 71, in _run_scenario_once getattr(scenario_inst, method_name)(**scenario_kwargs) File "/usr/local/lib/python2.7/dist-packages/rally/plugins/openstack/scenarios/nova/security_group.py", line 167, in run self._delete_server(server) File "/usr/local/lib/python2.7/dist-packages/rally/plugins/openstack/scenarios/nova/utils.py", line 451, in _delete_server check_interval=CONF.openstack.nova_server_delete_poll_interval File "/usr/local/lib/python2.7/dist-packages/rally/task/utils.py", line 214, in wait_for_status resource = update_resource(resource) File "/usr/local/lib/python2.7/dist-packages/rally/task/utils.py", line 80, in _get_from_manager raise exceptions.GetResourceFailure(resource=resource, err=e) GetResourceFailure: Failed to get the resource <Server: s_rally_41596539_fMbobPdS>: The server has either erred or is incapable of performing the requested operation. (HTTP 500) (Request-ID: req-a9471015-e72e-42cf-acc3-8b361415d91e) Neutron: ------- port-creation: 2018-04-22 09:18:31.984 32139 DEBUG vmware_nsxlib.v3.client [req-7f2bece2-7bd3-4e25-a5be-9c363e77bd5b f8e81abca4094924831b9538389b8d5b c87b5a2bc4e649069e9cfa4bbc874663 - default default] REST call: POST api/v1/logical-ports. Headers: {'X-NSX-EUSER': u'f8e81abca4094924831b9538389b8d5b c87b5a2bc4e649069e9cfa4bbc874663 - default default', 'Content-Type': 'application/json', 'X-NSX-EREQID': 'req-7f2bece2-7bd3-4e25-a5be-9c363e77bd5b', 'Accept': 'application/json'}. Body: {"address_bindings": [{"ip_address": "100.1.124.13", "mac_address": "fa:16:3e:d4:3a:8f"}], "admin_state": "UP", "attachment": null, "description": "", "display_name": "", "logical_switch_id": "dfc6fa69-6df7-44db-ba43-68152104103f", "switching_profile_ids": [{"key": "SpoofGuardSwitchingProfile", "value": "0bb5dd95-1e3d-464e-a9d5-bd040bd91d9d"}], "tags": [{"scope": "os-neutron-port-id", "tag": "1954b052-bc2c-48ba-aa24-e8c96fac1723"}, {"scope": "os-project-id", "tag": "c87b5a2bc4e649069e9cfa4bbc874663"}, {"scope": "os-project-name", "tag": "c_rally_41596539_tkcSTCuv"}, {"scope": "os-api-version", "tag": "12.0.0.dev8294691"}, {"scope": "os-security-group", "tag": "a4d155d5-285a-4e9a-8982-04f6a242d3ed"}, {"scope": "os-security-group", "tag": "OS-Default-Section"}]} _rest_call /usr/lib/python2.7/dist-packages/vmware_nsxlib/v3/client.py:201 . . 2018-04-22 09:18:32.278 32139 INFO neutron.wsgi [req-7f2bece2-7bd3-4e25-a5be-9c363e77bd5b f8e81abca4094924831b9538389b8d5b c87b5a2bc4e649069e9cfa4bbc874663 - default default] 10.127.108.239,10.127.108.237 "POST /v2.0/ports HTTP/1.1" status: 201 len: 983 time: 2.8035491 port-deletion: 2018-04-22 09:18:55.552 32135 INFO neutron.wsgi [req-774309da- 6fc6-4521-aed2-37f2eb01b5b4 f8e81abca4094924831b9538389b8d5b c87b5a2bc4e649069e9cfa4bbc874663 - default default] 10.127.108.239,10.127.108.237 "DELETE /v2.0/ports/1954b052-bc2c-48ba- aa24-e8c96fac1723 HTTP/1.1" status: 204 len: 168 time: 2.4448841 Nova: ---- 2018-04-22 09:18:55.539 19221 DEBUG oslo_concurrency.lockutils [req-824643af-4107-46a9-ba5a-950fcec27ba4 8d478ca1fa274601a50f8d2a38c5e70f 69fad6d3240645b5854b52bca7e9b85e - default default] Lock "9ab78fdd-5798-4426-baf2-2706d0d69345" acqu ired by "nova.context.get_or_set_cached_cell_and_set_connections" :: waited 0.000s inner /usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:276 2018-04-22 09:18:55.541 19221 DEBUG oslo_concurrency.lockutils [req-824643af-4107-46a9-ba5a-950fcec27ba4 8d478ca1fa274601a50f8d2a38c5e70f 69fad6d3240645b5854b52bca7e9b85e - default default] Lock "9ab78fdd-5798-4426-baf2-2706d0d69345" rele ased by "nova.context.get_or_set_cached_cell_and_set_connections" :: held 0.002s inner /usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:288 2018-04-22 09:18:55.662 19221 DEBUG nova.policy [req-824643af-4107-46a9-ba5a-950fcec27ba4 8d478ca1fa274601a50f8d2a38c5e70f 69fad6d3240645b5854b52bca7e9b85e - default default] Policy check for os_compute_api:os-extended-server-attributes f ailed with credentials {'service_roles': [], 'user_id': u'8d478ca1fa274601a50f8d2a38c5e70f', 'roles': [u'_member_'], 'user_domain_id': u'default', 'service_project_id': None, 'service_user_id': None, 'service_user_domain_id': None, 'servi ce_project_domain_id': None, 'is_admin_project': False, 'is_admin': False, 'project_id': u'69fad6d3240645b5854b52bca7e9b85e', 'project_domain_id': u'default'} authorize /usr/lib/python2.7/dist-packages/nova/policy.py:168 2018-04-22 09:18:55.774 19225 DEBUG neutronclient.v2_0.client [req-a9471015-e72e-42cf-acc3-8b361415d91e f8e81abca4094924831b9538389b8d5b c87b5a2bc4e649069e9cfa4bbc874663 - default default] Error message: {"NeutronError": {"message": "Port 1954b052-bc2c-48ba-aa24-e8c96fac1723 could not be found.", "type": "PortNotFound", "detail": ""}} _handle_fault_response /usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py:259 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack [req-a9471015-e72e-42cf-acc3-8b361415d91e f8e81abca4094924831b9538389b8d5b c87b5a2bc4e649069e9cfa4bbc874663 - default default] Caught error: Port 1954b052-bc2c-48ba-aa24-e8c96fac1723 could not be found. Neutron server returns request_ids: ['req-35436010-4faa-42de-ab84-a783f48b4f92']: PortNotFoundClient: Port 1954b052-bc2c-48ba-aa24-e8c96fac1723 could not be found. Neutron server returns request_ids: ['req-35436010-4faa-42de-ab84-a783f48b4f92'] 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack Traceback (most recent call last): 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack File "/usr/lib/python2.7/dist-packages/nova/api/openstack/__init__.py", line 82, in __call__ 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack return req.get_response(self.application) 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1316, in send 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack application, catch_exc_info=False) 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1280, in call_application 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack app_iter = application(self.environ, start_response) 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 131, in __call__ 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack resp = self.call_func(req, *args, **self.kwargs) 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 196, in call_func 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack return self.func(req, *args, **kwargs) 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack File "/usr/lib/python2.7/dist-packages/nova/api/openstack/requestlog.py", line 92, in __call__ 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack self._log_req(req, res, start) 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack self.force_reraise() 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack six.reraise(self.type_, self.value, self.tb) 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack File "/usr/lib/python2.7/dist-packages/nova/api/openstack/requestlog.py", line 87, in __call__ 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack res = req.get_response(self.application) 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1316, in send 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack application, catch_exc_info=False) 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1280, in call_application 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack app_iter = application(self.environ, start_response) 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 145, in __call__ 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack return resp(environ, start_response) 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 131, in __call__ 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack resp = self.call_func(req, *args, **self.kwargs) 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 196, in call_func 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack return self.func(req, *args, **kwargs) 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack File "/usr/lib/python2.7/dist-packages/osprofiler/web.py", line 112, in __call__ 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack return request.get_response(self.application) 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1316, in send 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack application, catch_exc_info=False) 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1280, in call_application 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack app_iter = application(self.environ, start_response) 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 131, in __call__ 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack resp = self.call_func(req, *args, **self.kwargs) 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 196, in call_func 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack return self.func(req, *args, **kwargs) 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack File "/usr/lib/python2.7/dist-packages/keystonemiddleware/auth_token/__init__.py", line 337, in __call__ 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack response = req.get_response(self._app) 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1316, in send 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack application, catch_exc_info=False) 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1280, in call_application 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack app_iter = application(self.environ, start_response) 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 145, in __call__ 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack return resp(environ, start_response) 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 145, in __call__ 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack return resp(environ, start_response) 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack File "/usr/lib/python2.7/dist-packages/routes/middleware.py", line 141, in __call__ 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack response = self.app(environ, start_response) 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 145, in __call__ 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack return resp(environ, start_response) 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 131, in __call__ 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack resp = self.call_func(req, *args, **self.kwargs) 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 196, in call_func 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack return self.func(req, *args, **kwargs) 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack File "/usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py", line 580, in __call__ 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack content_type, body, accept) 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack File "/usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py", line 654, in _process_stack 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack request, action_args) 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack File "/usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py", line 526, in process_extensions 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack **action_args) 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack File "/usr/lib/python2.7/dist-packages/nova/api/openstack/compute/security_groups.py", line 534, in show 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack return self._show(req, resp_obj) 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack File "/usr/lib/python2.7/dist-packages/nova/api/openstack/compute/security_groups.py", line 530, in _show 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack self._extend_servers(req, [resp_obj.obj['server']]) 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack File "/usr/lib/python2.7/dist-packages/nova/api/openstack/compute/security_groups.py", line 512, in _extend_servers 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack servers)) 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack File "/usr/lib/python2.7/dist-packages/nova/network/security_group/neutron_driver.py", line 376, in get_instances_security_groups_bindings 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack ports = self._get_ports_from_server_list(servers, neutron) 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack File "/usr/lib/python2.7/dist-packages/nova/network/security_group/neutron_driver.py", line 335, in _get_ports_from_server_list 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack ports.extend(neutron.list_ports(**search_opts).get('ports')) 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 114, in wrapper 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack ret = obj(*args, **kwargs) 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 786, in list_ports 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack **_params) 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 114, in wrapper 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack ret = obj(*args, **kwargs) 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 369, in list 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack for r in self._pagination(collection, path, **params): 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 384, in _pagination 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack res = self.get(path, params=params) 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 114, in wrapper 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack ret = obj(*args, **kwargs) 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 354, in get 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack headers=headers, params=params) 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 114, in wrapper 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack ret = obj(*args, **kwargs) 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 331, in retry_request 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack headers=headers, params=params) 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 114, in wrapper 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack ret = obj(*args, **kwargs) 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 294, in do_request 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack self._handle_fault_response(status_code, replybody, resp) 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 114, in wrapper 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack ret = obj(*args, **kwargs) 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 269, in _handle_fault_response 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack exception_handler_v20(status_code, error_body) 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 93, in exception_handler_v20 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack request_ids=request_ids) 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack PortNotFoundClient: Port 1954b052-bc2c-48ba-aa24-e8c96fac1723 could not be found. 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack Neutron server returns request_ids: ['req-35436010-4faa-42de-ab84-a783f48b4f92'] 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack 2018-04-22 09:18:55.819 19225 INFO nova.api.openstack [req-a9471015-e72e-42cf-acc3-8b361415d91e f8e81abca4094924831b9538389b8d5b c87b5a2bc4e649069e9cfa4bbc874663 - default default] http://192.168.20.101:8774/v2.1/c87b5a2bc4e649069e9cfa4bbc874663/servers/3e714b48-5193-4b13-b6c9-3374488b2e50 returned with HTTP 500 2018-04-22 09:18:55.822 19225 DEBUG nova.api.openstack.wsgi [req-a9471015-e72e-42cf-acc3-8b361415d91e f8e81abca4094924831b9538389b8d5b c87b5a2bc4e649069e9cfa4bbc874663 - default default] Returning 500 to user: The server has either erred or is incapable of performing the requested operation. __call__ /usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py:1064 At a deeper looks we see: After looking more into the logs and the flow, here are the relevant log lines that explain the issue, and indicate this is an exception not being handled in Nova (see here https://github.com/openstack/nova/blob/master/nova/network/security_group/neutron_driver.py#L378), note the timestamps: Port ID: 1954b052-bc2c-48ba-aa24-e8c96fac1723 Nova instance UUID: 3e714b48-5193-4b13-b6c9-3374488b2e50 Neutron logs ============= Port deletion: 2018-04-22 09:18:55.552 32135 INFO neutron.wsgi [req-774309da-6fc6-4521-aed2-37f2eb01b5b4 f8e81abca4094924831b9538389b8d5b c87b5a2bc4e649069e9cfa4bbc874663 - default default] 10.127.108.239,10.127.108.237 "DELETE /v2.0/ports/1954b052-bc2c-48ba-aa24-e8c96fac1723 HTTP/1.1" status: 204 len: 168 time: 2.4448841 GET ports failure: 2018-04-22 09:18:55.770 31303 INFO neutron.wsgi [req-35436010-4faa-42de-ab84-a783f48b4f92 f8e81abca4094924831b9538389b8d5b c87b5a2bc4e649069e9cfa4bbc874663 - default default] 10.127.108.236,10.127.108.237 "GET /v2.0/ports?device_id=3e714b48-5193-4b13-b6c9-3374488b2e50 HTTP/1.1" status: 404 len: 333 time: 0.5505300 Nova log ======== 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack PortNotFoundClient: Port 1954b052-bc2c-48ba-aa24-e8c96fac1723 could not be found. 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack Neutron server returns request_ids: ['req-35436010-4faa-42de-ab84-a783f48b4f92'] 2018-04-22 09:18:55.778 19225 ERROR nova.api.openstack 2018-04-22 09:18:55.819 19225 INFO nova.api.openstack [req-a9471015-e72e-42cf-acc3-8b361415d91e f8e81abca4094924831b9538389b8d5b c87b5a2bc4e649069e9cfa4bbc874663 - default default] http://192.168.20.101:8774/v2.1/c87b5a2bc4e649069e9cfa4bbc874663/servers/3e714b48-5193-4b13-b6c9-3374488b2e50 returned with HTTP 500 If there is a exception in self._get_secgroups_from_port_list should the security groups returned be []? Or a better exception raised. This is due to a race in the request in the retrieing the security groups and the deletion of the instance. The instance does not exist at this stage... The problem is discovered in stable queens To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1767058/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp