I hit this. I can reproduce it almost every time on my env using linuxbrige+vxlan. The nova trace is:
2016-07-02 16:20:52.670 9993 ERROR nova.api.openstack.extensions [req-bf41dac1-8fc0-4fd6-9a35-d754cea79057 9a0be6e4b8bf4cadb4a43401696fec19 48935f9a5ed84703973c70dd70859b7f - - -] Unexpected exception in API method 2016-07-02 16:20:52.670 9993 ERROR nova.api.openstack.extensions Traceback (most recent call last): 2016-07-02 16:20:52.670 9993 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/site-packages/nova/api/openstack/extensions.py", line 478, in wrapped 2016-07-02 16:20:52.670 9993 ERROR nova.api.openstack.extensions return f(*args, **kwargs) 2016-07-02 16:20:52.670 9993 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/site-packages/nova/api/openstack/compute/floating_ips.py", line 173, in delete 2016-07-02 16:20:52.670 9993 ERROR nova.api.openstack.extensions context, instance, floating_ip) 2016-07-02 16:20:52.670 9993 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 1527, in disassociate_and_release_floating_ip 2016-07-02 16:20:52.670 9993 ERROR nova.api.openstack.extensions raise_if_associated=False) 2016-07-02 16:20:52.670 9993 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 1536, in _release_floating_ip 2016-07-02 16:20:52.670 9993 ERROR nova.api.openstack.extensions client.delete_floatingip(fip['id']) 2016-07-02 16:20:52.670 9993 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 102, in with_params 2016-07-02 16:20:52.670 9993 ERROR nova.api.openstack.extensions ret = self.function(instance, *args, **kwargs) 2016-07-02 16:20:52.670 9993 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 751, in delete_floatingip 2016-07-02 16:20:52.670 9993 ERROR nova.api.openstack.extensions return self.delete(self.floatingip_path % (floatingip)) 2016-07-02 16:20:52.670 9993 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 289, in delete 2016-07-02 16:20:52.670 9993 ERROR nova.api.openstack.extensions headers=headers, params=params) 2016-07-02 16:20:52.670 9993 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 270, in retry_request 2016-07-02 16:20:52.670 9993 ERROR nova.api.openstack.extensions headers=headers, params=params) 2016-07-02 16:20:52.670 9993 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 211, in do_request 2016-07-02 16:20:52.670 9993 ERROR nova.api.openstack.extensions self._handle_fault_response(status_code, replybody) 2016-07-02 16:20:52.670 9993 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 185, in _handle_fault_response 2016-07-02 16:20:52.670 9993 ERROR nova.api.openstack.extensions exception_handler_v20(status_code, des_error_body) 2016-07-02 16:20:52.670 9993 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 70, in exception_handler_v20 2016-07-02 16:20:52.670 9993 ERROR nova.api.openstack.extensions status_code=status_code) 2016-07-02 16:20:52.670 9993 ERROR nova.api.openstack.extensions PortNotFoundClient: Port f4f11381-dc3b-41b2-94ca-4a9f494c0372 could not be found. 2016-07-02 16:20:52.670 9993 ERROR nova.api.openstack.extensions 2 operation occurs: the VM deletion (that triggers the Neutron port deletion) and the floating ip deletion. Nova sends a request to Neutron to delete the floating IP. When a floating IP is deleted Neutron will get the port associated with the floating ip to send a network change event notification to Nova. The get port fails with PortNotFound because in the meanwhile the Neutron port that the VM was using, has been deleted. The floating ip request fails because Neutron sends back to Nova a PortNotFound error. ** Also affects: neutron Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1586931 Title: TestServerBasicOps: Test fails when deleting server and floating ip almost at the same time Status in neutron: New Status in OpenStack Compute (nova): Incomplete Status in tempest: In Progress Bug description: In tempest.scenario.test_server_basic_ops.TestServerBasicOps.test_server_basic_ops, after last step: self.servers_client.delete_server(self.instance['id']) it doesn't wait for the server to be deleted, and then deletes the floating ip immediately in the clean up, this will cause faiure: Here is the partial log: 2016-05-29 21:51:29.499 29791 INFO tempest.lib.common.rest_client [req-c3588ac4-21ca-47c3-bdb1-62088efd7a8b ] Request (TestServerBasicOps:test_server_basic_ops): 204 DELETE https://<url>:8774/v2/159886ce087a4f8fbfbcab14947d96b1/servers/6d44763b-ea79-4b5b-b57e-714191802c7c 0.465s 2016-05-29 21:51:29.499 29791 DEBUG tempest.lib.common.rest_client [req-c3588ac4-21ca-47c3-bdb1-62088efd7a8b ] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'} Body: None Response - Headers: {'status': '204', 'content-length': '0', 'content-location': 'https://<url>:8774/v2/159886ce087a4f8fbfbcab14947d96b1/servers/6d44763b-ea79-4b5b-b57e-714191802c7c', 'date': 'Mon, 30 May 2016 02:51:29 GMT', 'x-compute-request-id': 'req-c3588ac4-21ca-47c3-bdb1-62088efd7a8b', 'content-type': 'application/json', 'connection': 'close'} Body: _log_request_full tempest/lib/common/rest_client.py:422 2016-05-29 21:51:30.410 29791 INFO tempest.lib.common.rest_client [req-db2323f5-3d58-4fd7-ae51-44f5525c6689 ] Request (TestServerBasicOps:_run_cleanups): 500 DELETE https://<url>:8774/v2/159886ce087a4f8fbfbcab14947d96b1/os-floating-ips/948912f6-ce03-4856-922b-59c4f16d3740 0.910s 2016-05-29 21:51:30.410 29791 DEBUG tempest.lib.common.rest_client [req-db2323f5-3d58-4fd7-ae51-44f5525c6689 ] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'} Body: None Response - Headers: {'status': '500', 'content-length': '224', 'content-location': 'https://<url>:8774/v2/159886ce087a4f8fbfbcab14947d96b1/os-floating-ips/948912f6-ce03-4856-922b-59c4f16d3740', 'date': 'Mon, 30 May 2016 02:51:30 GMT', 'x-compute-request-id': 'req-db2323f5-3d58-4fd7-ae51-44f5525c6689', 'content-type': 'application/json; charset=UTF-8', 'connection': 'close'} Body: {"computeFault": {"message": "Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.\n<class 'neutronclient.common.exceptions.PortNotFoundClient'>", "code": 500}} To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1586931/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp