Public bug reported: We're running into an issue in our CI with Atmosphere where we frequently see failures when a router port is removed from an interface, the traceback is the following:
============================== 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource [None req-e5d08cdd-28e6-4231-a50c-7eafc1b8f942 70fc3b55af8c4386b80207dad11db5da dcec54844db44eedbd9667951a5ceb6b - - - -] remove_router_interface failed: No details.: ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router_Port with name=lrp-7e0debbb-893c-420a-8569-d8fb98e6a16e 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource Traceback (most recent call last): 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/neutron/api/v2/resource.py", line 98, in resource 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource result = method(request=request, **args) 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/neutron_lib/db/api.py", line 140, in wrapped 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource with excutils.save_and_reraise_exception(): 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__ 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource self.force_reraise() 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource raise self.value 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/neutron_lib/db/api.py", line 138, in wrapped 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource return f(*args, **kwargs) 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/oslo_db/api.py", line 144, in wrapper 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource with excutils.save_and_reraise_exception() as ectxt: 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__ 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource self.force_reraise() 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource raise self.value 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/oslo_db/api.py", line 142, in wrapper 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource return f(*args, **kwargs) 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/neutron_lib/db/api.py", line 186, in wrapped 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource with excutils.save_and_reraise_exception(): 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__ 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource self.force_reraise() 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource raise self.value 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/neutron_lib/db/api.py", line 184, in wrapped 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource return f(*dup_args, **dup_kwargs) 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/neutron/api/v2/base.py", line 253, in _handle_action 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource ret_value = getattr(self._plugin, name)(*arg_list, **kwargs) 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/neutron/services/ovn_l3/plugin.py", line 260, in remove_router_interface 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource with excutils.save_and_reraise_exception(): 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__ 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource self.force_reraise() 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource raise self.value 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/neutron/services/ovn_l3/plugin.py", line 257, in remove_router_interface 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource self._ovn_client.delete_router_port( 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1904, in delete_router_port 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource with self._nb_idl.transaction(check_error=True) as txn: 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource next(self.gen) 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource with super(OvsdbNbOvnIdl, self).transaction(*args, **kwargs) as t: 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource next(self.gen) 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/ovsdbapp/api.py", line 104, in transaction 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource with self.create_transaction( 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/ovsdbapp/api.py", line 61, in __exit__ 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource self.result = self.commit() 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource raise result.ex 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 118, in run 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource txn.results.put(txn.do_commit()) 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 92, in do_commit 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource command.run_idl(txn) 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 341, in run_idl 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource entity = self.api.lookup(self.table, self.entity) 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 181, in lookup 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource return self._lookup(table, record) 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 228, in _lookup 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource row = idlutils.row_by_value(self, rl.table, rl.column, record) 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 114, in row_by_value 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource raise RowNotFound(table=table, col=column, match=match) 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router_Port with name=lrp-7e0debbb-893c-420a-8569-d8fb98e6a16e 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource [00m 2024-04-03 21:13:09.807 10 INFO neutron.wsgi [None req-e5d08cdd-28e6-4231-a50c-7eafc1b8f942 70fc3b55af8c4386b80207dad11db5da dcec54844db44eedbd9667951a5ceb6b - - - -] 10.0.0.73 "PUT /v2.0/routers/1a3225bc-725d-400f-b9ef-3ffe7d4fb8d8/remove_router_interface HTTP/1.1" status: 500 len: 363 time: 2.0234911[00m ============================== I can't tell what's going on exactly, but it seems like for some reason the `lrp` is missing when we try and remove it. The add seems to happen just fine though: ============================== 2024-04-03 21:12:58.396 9 INFO neutron.wsgi [None req-0b71c77f-3844-4549-af1e-541f344f302e 70fc3b55af8c4386b80207dad11db5da dcec54844db44eedbd9667951a5ceb6b - - - -] 10.0.0.73 "PUT /v2.0/routers/1a3225bc-725d-400f-b9ef-3ffe7d4fb8d8/add_router_interface HTTP/1.1" status: 200 len: 503 time: 3.1546130[00m ============================== https://logs.atmosphere.dev/61/1061/46e91a5e83194215e35f7209ff92bf86597b4cd6/check/atmosphere- molecule-aio-ovn/5f98342/ubuntu-jammy/pod-logs/openstack/neutron- server-746d84b879-6q8kt/neutron-server.txt Since this is a CI environment, in order to reduce the resources we consume, we limit neutron to 2 workers. Is it possible that it's not keeping up with the events coming in from OVN and we end up in this mismatch? ** Affects: neutron Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/2060163 Title: [ovn] race condition with add/remove router interface Status in neutron: New Bug description: We're running into an issue in our CI with Atmosphere where we frequently see failures when a router port is removed from an interface, the traceback is the following: ============================== 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource [None req-e5d08cdd-28e6-4231-a50c-7eafc1b8f942 70fc3b55af8c4386b80207dad11db5da dcec54844db44eedbd9667951a5ceb6b - - - -] remove_router_interface failed: No details.: ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router_Port with name=lrp-7e0debbb-893c-420a-8569-d8fb98e6a16e 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource Traceback (most recent call last): 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/neutron/api/v2/resource.py", line 98, in resource 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource result = method(request=request, **args) 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/neutron_lib/db/api.py", line 140, in wrapped 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource with excutils.save_and_reraise_exception(): 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__ 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource self.force_reraise() 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource raise self.value 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/neutron_lib/db/api.py", line 138, in wrapped 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource return f(*args, **kwargs) 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/oslo_db/api.py", line 144, in wrapper 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource with excutils.save_and_reraise_exception() as ectxt: 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__ 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource self.force_reraise() 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource raise self.value 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/oslo_db/api.py", line 142, in wrapper 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource return f(*args, **kwargs) 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/neutron_lib/db/api.py", line 186, in wrapped 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource with excutils.save_and_reraise_exception(): 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__ 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource self.force_reraise() 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource raise self.value 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/neutron_lib/db/api.py", line 184, in wrapped 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource return f(*dup_args, **dup_kwargs) 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/neutron/api/v2/base.py", line 253, in _handle_action 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource ret_value = getattr(self._plugin, name)(*arg_list, **kwargs) 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/neutron/services/ovn_l3/plugin.py", line 260, in remove_router_interface 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource with excutils.save_and_reraise_exception(): 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__ 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource self.force_reraise() 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource raise self.value 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/neutron/services/ovn_l3/plugin.py", line 257, in remove_router_interface 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource self._ovn_client.delete_router_port( 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1904, in delete_router_port 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource with self._nb_idl.transaction(check_error=True) as txn: 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource next(self.gen) 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource with super(OvsdbNbOvnIdl, self).transaction(*args, **kwargs) as t: 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource next(self.gen) 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/ovsdbapp/api.py", line 104, in transaction 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource with self.create_transaction( 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/ovsdbapp/api.py", line 61, in __exit__ 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource self.result = self.commit() 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource raise result.ex 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 118, in run 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource txn.results.put(txn.do_commit()) 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 92, in do_commit 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource command.run_idl(txn) 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 341, in run_idl 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource entity = self.api.lookup(self.table, self.entity) 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 181, in lookup 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource return self._lookup(table, record) 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 228, in _lookup 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource row = idlutils.row_by_value(self, rl.table, rl.column, record) 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource File "/var/lib/openstack/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 114, in row_by_value 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource raise RowNotFound(table=table, col=column, match=match) 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router_Port with name=lrp-7e0debbb-893c-420a-8569-d8fb98e6a16e 2024-04-03 21:13:09.804 10 ERROR neutron.api.v2.resource [00m 2024-04-03 21:13:09.807 10 INFO neutron.wsgi [None req-e5d08cdd-28e6-4231-a50c-7eafc1b8f942 70fc3b55af8c4386b80207dad11db5da dcec54844db44eedbd9667951a5ceb6b - - - -] 10.0.0.73 "PUT /v2.0/routers/1a3225bc-725d-400f-b9ef-3ffe7d4fb8d8/remove_router_interface HTTP/1.1" status: 500 len: 363 time: 2.0234911[00m ============================== I can't tell what's going on exactly, but it seems like for some reason the `lrp` is missing when we try and remove it. The add seems to happen just fine though: ============================== 2024-04-03 21:12:58.396 9 INFO neutron.wsgi [None req-0b71c77f-3844-4549-af1e-541f344f302e 70fc3b55af8c4386b80207dad11db5da dcec54844db44eedbd9667951a5ceb6b - - - -] 10.0.0.73 "PUT /v2.0/routers/1a3225bc-725d-400f-b9ef-3ffe7d4fb8d8/add_router_interface HTTP/1.1" status: 200 len: 503 time: 3.1546130[00m ============================== https://logs.atmosphere.dev/61/1061/46e91a5e83194215e35f7209ff92bf86597b4cd6/check/atmosphere- molecule-aio-ovn/5f98342/ubuntu-jammy/pod-logs/openstack/neutron- server-746d84b879-6q8kt/neutron-server.txt Since this is a CI environment, in order to reduce the resources we consume, we limit neutron to 2 workers. Is it possible that it's not keeping up with the events coming in from OVN and we end up in this mismatch? To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/2060163/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp