** Changed in: nova/ussuri Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1907522
Title: test_*_with_qos_min_bw_allocation fails in the nova-multi-cell job with: nova.exception.MigrationPreCheckError: Migration pre-check error: Failed to create port bindings for host <host> Status in OpenStack Compute (nova): Fix Released Status in OpenStack Compute (nova) ussuri series: Fix Released Status in OpenStack Compute (nova) victoria series: Fix Released Bug description: Seen in the gate on the master branch, the test_migrate_with_qos_min_bw_allocation and test_resize_with_qos_min_bw_allocation tests that were recently added to tempest [1] are failing in the nova-multi-cell job with: "ERROR oslo_messaging.rpc.server nova.exception.MigrationPreCheckError: Migration pre-check error: Failed to create port bindings for host ubuntu-focal-ovh- bhs1-0022118086" with the following traceback in screen-n-super-cond.txt [2]: Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server Traceback (most recent call last): Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server File "/opt/stack/nova/nova/conductor/tasks/cross_cell_migrate.py", line 266, in _create_port_bindings Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server self._bindings_by_port_id = self.network_api.bind_ports_to_host( Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server File "/opt/stack/nova/nova/network/neutron.py", line 1352, in bind_ports_to_host Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server raise exception.PortBindingFailed(port_id=port_id) Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server nova.exception.PortBindingFailed: Binding failed for port d37d5be3-df95-4842-bddf-2058d8e452fc, please check neutron logs for more information. Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server During handling of the above exception, another exception occurred: Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server Traceback (most recent call last): Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server File "/opt/stack/nova/nova/conductor/manager.py", line 354, in _cold_migrate Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server task.execute() Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server File "/opt/stack/nova/nova/conductor/tasks/base.py", line 26, in wrap Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server self.rollback(ex) Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server self.force_reraise() Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.8/dist-packages/six.py", line 703, in reraise Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server raise value Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server File "/opt/stack/nova/nova/conductor/tasks/base.py", line 23, in wrap Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server return original(self) Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server File "/opt/stack/nova/nova/conductor/tasks/base.py", line 40, in execute Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server return self._execute() Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server File "/opt/stack/nova/nova/conductor/tasks/migrate.py", line 295, in _execute Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server task.execute() Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server File "/opt/stack/nova/nova/conductor/tasks/base.py", line 26, in wrap Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server self.rollback(ex) Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server self.force_reraise() Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.8/dist-packages/six.py", line 703, in reraise Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server raise value Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server File "/opt/stack/nova/nova/conductor/tasks/base.py", line 23, in wrap Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server return original(self) Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server File "/opt/stack/nova/nova/conductor/tasks/base.py", line 40, in execute Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server return self._execute() Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server File "/opt/stack/nova/nova/conductor/tasks/cross_cell_migrate.py", line 838, in _execute Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server target_cell_migration = self._prep_resize_at_dest( Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server File "/opt/stack/nova/nova/conductor/tasks/cross_cell_migrate.py", line 727, in _prep_resize_at_dest Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server target_cell_migration_context = verify_task.execute() Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server File "/opt/stack/nova/nova/conductor/tasks/base.py", line 26, in wrap Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server self.rollback(ex) Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server self.force_reraise() Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.8/dist-packages/six.py", line 703, in reraise Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server raise value Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server File "/opt/stack/nova/nova/conductor/tasks/base.py", line 23, in wrap Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server return original(self) Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server File "/opt/stack/nova/nova/conductor/tasks/base.py", line 40, in execute Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server return self._execute() Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server File "/opt/stack/nova/nova/conductor/tasks/cross_cell_migrate.py", line 327, in _execute Dec 08 19:11:12.292344 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server self._create_port_bindings() Dec 08 19:11:12.297758 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server File "/opt/stack/nova/nova/conductor/tasks/cross_cell_migrate.py", line 269, in _create_port_bindings Dec 08 19:11:12.297758 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server raise exception.MigrationPreCheckError(reason=_( Dec 08 19:11:12.297758 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server nova.exception.MigrationPreCheckError: Migration pre-check error: Failed to create port bindings for host ubuntu-focal-ovh-bhs1-0022118086 Dec 08 19:11:12.297758 ubuntu-focal-ovh-bhs1-0022118085 nova-conductor[93226]: ERROR oslo_messaging.rpc.server and corresponding neutron failure: "ERROR neutron.plugins.ml2.managers neutron_lib.exceptions.placement.UnknownResourceProvider: No such resource provider known by Neutron" with traceback in screen-q-svc.txt [3] : Dec 08 19:11:11.521492 ubuntu-focal-ovh-bhs1-0022118085 neutron-server[85124]: ERROR neutron.plugins.ml2.managers [req-1d6e98ea-4f98-4a73-a4f7-470561c7e03b req-0389a6b9-c4f4-448e-baa9-8f31bf231926 service neutron] Failed to bind port d37d5be3-df95-4842-bddf-2058d8e452fc on host ubuntu-focal-ovh-bhs1-0022118086 allocated on resource provider c7942897-51a0-56e7-86e1-4727af85911d, because no mechanism driver reports being responsible Dec 08 19:11:11.526334 ubuntu-focal-ovh-bhs1-0022118085 neutron-server[85124]: ERROR neutron.plugins.ml2.managers [req-1d6e98ea-4f98-4a73-a4f7-470561c7e03b req-0389a6b9-c4f4-448e-baa9-8f31bf231926 service neutron] Mechanism driver openvswitch failed in bind_port: neutron_lib.exceptions.placement.UnknownResourceProvider: No such resource provider known by Neutron: c7942897-51a0-56e7-86e1-4727af85911d Dec 08 19:11:11.526334 ubuntu-focal-ovh-bhs1-0022118085 neutron-server[85124]: ERROR neutron.plugins.ml2.managers Traceback (most recent call last): Dec 08 19:11:11.526334 ubuntu-focal-ovh-bhs1-0022118085 neutron-server[85124]: ERROR neutron.plugins.ml2.managers File "/opt/stack/neutron/neutron/plugins/ml2/managers.py", line 923, in _bind_port_level Dec 08 19:11:11.526334 ubuntu-focal-ovh-bhs1-0022118085 neutron-server[85124]: ERROR neutron.plugins.ml2.managers drivers=[self._infer_driver_from_allocation( Dec 08 19:11:11.526334 ubuntu-focal-ovh-bhs1-0022118085 neutron-server[85124]: ERROR neutron.plugins.ml2.managers File "/opt/stack/neutron/neutron/plugins/ml2/managers.py", line 973, in _infer_driver_from_allocation Dec 08 19:11:11.526334 ubuntu-focal-ovh-bhs1-0022118085 neutron-server[85124]: ERROR neutron.plugins.ml2.managers raise place_exc.UnknownResourceProvider( Dec 08 19:11:11.526334 ubuntu-focal-ovh-bhs1-0022118085 neutron-server[85124]: ERROR neutron.plugins.ml2.managers neutron_lib.exceptions.placement.UnknownResourceProvider: No such resource provider known by Neutron: c7942897-51a0-56e7-86e1-4727af85911d Dec 08 19:11:11.526334 ubuntu-focal-ovh-bhs1-0022118085 neutron-server[85124]: ERROR neutron.plugins.ml2.managers [1] https://review.opendev.org/c/openstack/tempest/+/694539 [2] https://zuul.opendev.org/t/openstack/build/d7c15477b23e4a28b400c30277b3b3e3/log/controller/logs/screen-n-super-cond.txt?severity=4#4180 [3] https://zuul.opendev.org/t/openstack/build/d7c15477b23e4a28b400c30277b3b3e3/log/controller/logs/screen-q-svc.txt#72367 To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1907522/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp