Public bug reported: When there is router with subnet with enabled prefix delegation plugged and L3 agent is restarted it's failing with error like:
2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: 7ef0983f-60a4-4389-909c-785eb6c9a51e: neutron.privileged.agent.linux.ip_lib.IpAddressAlreadyExists: IP address fe80::f816:3eff:fec8:b9a8 already configured on qg-351b89d8-61. 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/neutron/agent/l3/agent.py", line 694, in _process_routers_if_compatible 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent self._process_router_if_compatible(router) 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/neutron/agent/l3/agent.py", line 540, in _process_router_if_compatible 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent self._process_updated_router(router) 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/neutron/agent/l3/agent.py", line 581, in _process_updated_router 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent ri.process() 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/neutron/common/utils.py", line 161, in call 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent self.logger(e) 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent self.force_reraise() 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent six.reraise(self.type_, self.value, self.tb) 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/six.py", line 693, in reraise 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent raise value 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/neutron/common/utils.py", line 158, in call 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent return func(*args, **kwargs) 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/neutron/agent/l3/router_info.py", line 1191, in process 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent self.process_external() 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/neutron/agent/l3/router_info.py", line 949, in process_external 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent self._process_external_gateway(ex_gw_port) 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/neutron/agent/l3/router_info.py", line 833, in _process_external_gateway 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent interface_name) 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py", line 328, in inner 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent return f(*args, **kwargs) 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/neutron/agent/linux/pd.py", line 144, in add_gw_interface 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent self._add_lla(router, bind_lla_with_mask) 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/neutron/agent/linux/pd.py", line 220, in _add_lla 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent 'link') 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/neutron/agent/linux/interface.py", line 208, in add_ipv6_addr 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent device.addr.add(str(net), scope) 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/neutron/agent/linux/ip_lib.py", line 508, in add 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent add_broadcast) 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/neutron/agent/linux/ip_lib.py", line 872, in add_ip_address 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent device, namespace, scope, broadcast) 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/neutron/privileged/agent/linux/ip_lib.py", line 50, in sync_inner 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent return input_func(*args, **kwargs) 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/oslo_privsep/priv_context.py", line 244, in _wrap 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent return self.channel.remote_call(name, args, kwargs) 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/oslo_privsep/daemon.py", line 203, in remote_call 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent raise exc_type(*result[2]) 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent neutron.privileged.agent.linux.ip_lib.IpAddressAlreadyExists: IP address fe80::f816:3eff:fec8:b9a8 already configured on qg-351b89d8-61. 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent To workaround this problem I had to e.g. remove qrouter-XXX namespace from the node and then restart L3 agent. In that case all was configured without this error. ** Affects: neutron Importance: Medium Status: Confirmed ** Tags: l3-dvr-backlog -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1892362 Title: Restarting L3 agent when PD is used fails due to IPAddressAlreadyExists error Status in neutron: Confirmed Bug description: When there is router with subnet with enabled prefix delegation plugged and L3 agent is restarted it's failing with error like: 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent [-] Failed to process compatible router: 7ef0983f-60a4-4389-909c-785eb6c9a51e: neutron.privileged.agent.linux.ip_lib.IpAddressAlreadyExists: IP address fe80::f816:3eff:fec8:b9a8 already configured on qg-351b89d8-61. 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/neutron/agent/l3/agent.py", line 694, in _process_routers_if_compatible 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent self._process_router_if_compatible(router) 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/neutron/agent/l3/agent.py", line 540, in _process_router_if_compatible 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent self._process_updated_router(router) 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/neutron/agent/l3/agent.py", line 581, in _process_updated_router 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent ri.process() 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/neutron/common/utils.py", line 161, in call 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent self.logger(e) 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent self.force_reraise() 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent six.reraise(self.type_, self.value, self.tb) 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/six.py", line 693, in reraise 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent raise value 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/neutron/common/utils.py", line 158, in call 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent return func(*args, **kwargs) 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/neutron/agent/l3/router_info.py", line 1191, in process 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent self.process_external() 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/neutron/agent/l3/router_info.py", line 949, in process_external 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent self._process_external_gateway(ex_gw_port) 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/neutron/agent/l3/router_info.py", line 833, in _process_external_gateway 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent interface_name) 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py", line 328, in inner 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent return f(*args, **kwargs) 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/neutron/agent/linux/pd.py", line 144, in add_gw_interface 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent self._add_lla(router, bind_lla_with_mask) 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/neutron/agent/linux/pd.py", line 220, in _add_lla 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent 'link') 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/neutron/agent/linux/interface.py", line 208, in add_ipv6_addr 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent device.addr.add(str(net), scope) 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/neutron/agent/linux/ip_lib.py", line 508, in add 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent add_broadcast) 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/neutron/agent/linux/ip_lib.py", line 872, in add_ip_address 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent device, namespace, scope, broadcast) 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/neutron/privileged/agent/linux/ip_lib.py", line 50, in sync_inner 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent return input_func(*args, **kwargs) 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/oslo_privsep/priv_context.py", line 244, in _wrap 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent return self.channel.remote_call(name, args, kwargs) 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/oslo_privsep/daemon.py", line 203, in remote_call 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent raise exc_type(*result[2]) 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent neutron.privileged.agent.linux.ip_lib.IpAddressAlreadyExists: IP address fe80::f816:3eff:fec8:b9a8 already configured on qg-351b89d8-61. 2020-08-20 12:54:33.390 487489 ERROR neutron.agent.l3.agent To workaround this problem I had to e.g. remove qrouter-XXX namespace from the node and then restart L3 agent. In that case all was configured without this error. To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1892362/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp