** Changed in: neutron Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1668410
Title: [SRU] Infinite loop trying to delete deleted HA router Status in Ubuntu Cloud Archive: Invalid Status in Ubuntu Cloud Archive mitaka series: Fix Released Status in neutron: Fix Released Status in OpenStack Security Advisory: Won't Fix Status in neutron package in Ubuntu: Invalid Status in neutron source package in Xenial: Fix Released Bug description: [Descriptoin] When deleting a router the logfile is filled up. See full log - http://paste.ubuntu.com/25429257/ I can see the error 'Error while deleting router c0dab368-5ac8-4996-88c9-f5d345a774a6' occured 3343386 times from _safe_router_removed() [1]: $ grep -r 'Error while deleting router c0dab368-5ac8-4996-88c9-f5d345a774a6' |wc -l 3343386 This _safe_router_removed() is invoked by L488 [2], if _safe_router_removed() goes wrong it will return False, then self._resync_router(update) [3] will make the code _safe_router_removed be run again and again. So we saw so many errors 'Error while deleting router XXXXX'. [1] https://github.com/openstack/neutron/blob/mitaka-eol/neutron/agent/l3/agent.py#L361 [2] https://github.com/openstack/neutron/blob/mitaka-eol/neutron/agent/l3/agent.py#L488 [3] https://github.com/openstack/neutron/blob/mitaka-eol/neutron/agent/l3/agent.py#L457 [Test Case] That's because race condition between neutron server and L3 agent, after neutron server deletes HA interfaces the L3 agent may sync a HA router without HA interface info (just need to trigger L708[1] after deleting HA interfaces and before deleting HA router). If we delete HA router at this time, this problem will happen. So test case we design is as below: 1, First update fixed package, and restart neutron-server by 'sudo service neutron-server restart' 2, Create ha_router neutron router-create harouter --ha=True 3, Delete ports associated with ha_router before deleting ha_router neutron router-port-list harouter |grep 'HA port' |awk '{print $2}' |xargs -l neutron port-delete neutron router-port-list harouter 4, Update ha_router to trigger l3-agent to update ha_router info without ha_port into self.router_info neutron router-update harouter --description=test 5, Delete ha_router this time neutron router-delete harouter [1] https://github.com/openstack/neutron/blob/mitaka- eol/neutron/db/l3_hamode_db.py#L708 [Regression Potential] The fixed patch [1] for neutron-server will no longer return ha_router which is missing ha_ports, so L488 will no longer have chance to call _safe_router_removed() for a ha_router, so the problem has been fundamentally fixed by this patch and no regression potential. Besides, this fixed patch has been in mitaka-eol branch now, and neutron-server mitaka package is based on neutron-8.4.0, so we need to backport it to xenial and mitaka. $ git tag --contains 8c77ee6b20dd38cc0246e854711cb91cffe3a069 mitaka-eol [1] https://review.openstack.org/#/c/440799/2/neutron/db/l3_hamode_db.py [2] https://github.com/openstack/neutron/blob/mitaka-eol/neutron/agent/l3/agent.py#L488 To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1668410/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp