We still hit this with 7a70217d78 applied, through another path, and can reproduce it on master.
7a70217d78 keeps the floating IP's revision row when the deleting worker cannot see the NAT, so that maintenance retries the delete. But the retry can run on a worker that cannot see the NAT either; it then drops the row, and the NAT stays in the OVN Northbound database for good. This happens whenever the maintenance worker holding the lock works from stale data. For example, the NB raft leader freezes long enough for the followers to elect a new leader, while the lock holder's inactivity probe has not fired yet: it stays attached to the old leader, keeps the lock and runs maintenance from its frozen copy of the NB, while the other units move to the new leader. A floating IP associated and deleted in that window has a NAT the stale worker cannot see, so it drops the revision row. Nothing brings the NAT back to maintenance, and a later floating IP with the same address on the same router loses N/S traffic, as described in this bug. The attached functional test (test_floating_ip_delete_stale_view_keeps_revision, test_maintenance.py) deletes a floating IP without its NAT, runs one maintenance pass that cannot see the NAT, and checks that the row is kept and that a normal pass then deletes the NAT. It fails on master (37519b260d). We also reproduced it end to end in a lab, on master and on 2026.1 with 7a70217d78 backported: 3-member NB cluster, three API units, leader frozen with SIGSTOP (the lock holder had a longer ovsdb_probe_interval so it is the last to notice; with equal settings it depends on timing). The NAT was left behind, and a new floating IP with the same address on the same router passed 0 of 10 pings; removing only the leftover NAT restored it. Proposed fix: keep a deleted resource's revision row as a tombstone for a configurable period (default 2 hours, a margin over how long a worker can act on a stale view) instead of dropping it when the OVN object is not seen; the first pass with an up-to-date view then cleans up. Done in one place (a session hook records the deletion of any standard-attribute resource; delete_revision() keeps those rows), it covers every resource type. A prototype makes the attached test pass; six existing functional tests that expect the row gone right after a delete would need updating. Happy to propose the test and the fix on Gerrit if this direction looks acceptable. ** Patch added: "Functional test: maintenance task drops revision row from stale data" https://bugs.launchpad.net/neutron/+bug/1987530/+attachment/6016570/+files/0001-OVN-Add-test-for-maintenance-with-a-stale-NB-view.patch -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1987530 Title: Duplicate external_ip in NAT table lead to loss of N/S connectivity To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1987530/+subscriptions -- ubuntu-bugs mailing list [email protected] https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
