Public bug reported: Happens in Kilo. Cannot test on other releases.
Steps to reproduce, create a router and dont add any port. The command 'neutron router-port-list' should return an empty set. As soon as you create the router, the namespace qrouter-<uuid> is present on the network node, but after a while it is purged. As soon as the namespace is not there anymore check the log file vpn-agent.log for the stacktrace. When a router has no ports the namespace is deleted from the network node. However this brakes the router updates and the file vpn-agent.log is flooded with this traces: 2016-04-21 16:22:17.771 23382 TRACE neutron.agent.l3.router_info Traceback (most recent call last): 2016-04-21 16:22:17.771 23382 TRACE neutron.agent.l3.router_info File "/usr/lib/python2.7/dist-packages/neutron/common/utils.py", line 343, in call 2016-04-21 16:22:17.771 23382 TRACE neutron.agent.l3.router_info return func(*args, **kwargs) 2016-04-21 16:22:17.771 23382 TRACE neutron.agent.l3.router_info File "/usr/lib/python2.7/dist-packages/neutron/agent/l3/router_info.py", line 628, in process 2016-04-21 16:22:17.771 23382 TRACE neutron.agent.l3.router_info self._process_internal_ports() 2016-04-21 16:22:17.771 23382 TRACE neutron.agent.l3.router_info File "/usr/lib/python2.7/dist-packages/neutron/agent/l3/router_info.py", line 404, in _process_internal_ports 2016-04-21 16:22:17.771 23382 TRACE neutron.agent.l3.router_info existing_devices = self._get_existing_devices() 2016-04-21 16:22:17.771 23382 TRACE neutron.agent.l3.router_info File "/usr/lib/python2.7/dist-packages/neutron/agent/l3/router_info.py", line 328, in _get_existing_devices 2016-04-21 16:22:17.771 23382 TRACE neutron.agent.l3.router_info ip_devs = ip_wrapper.get_devices(exclude_loopback=True) 2016-04-21 16:22:17.771 23382 TRACE neutron.agent.l3.router_info File "/usr/lib/python2.7/dist-packages/neutron/agent/linux/ip_lib.py", line 102, in get_devices 2016-04-21 16:22:17.771 23382 TRACE neutron.agent.l3.router_info log_fail_as_error=self.log_fail_as_error 2016-04-21 16:22:17.771 23382 TRACE neutron.agent.l3.router_info File "/usr/lib/python2.7/dist-packages/neutron/agent/linux/utils.py", line 137, in execute 2016-04-21 16:22:17.771 23382 TRACE neutron.agent.l3.router_info raise RuntimeError(m) 2016-04-21 16:22:17.771 23382 TRACE neutron.agent.l3.router_info RuntimeError: 2016-04-21 16:22:17.771 23382 TRACE neutron.agent.l3.router_info Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-8fc0f640-35bb-4d0b-bbbd-80c22be0e762', 'find', '/sys/class/net', '-maxdepth', '1', '-type', 'l', '-printf', '%f '] 2016-04-21 16:22:17.771 23382 TRACE neutron.agent.l3.router_info Exit code: 1 2016-04-21 16:22:17.771 23382 TRACE neutron.agent.l3.router_info Stdin: 2016-04-21 16:22:17.771 23382 TRACE neutron.agent.l3.router_info Stdout: 2016-04-21 16:22:17.771 23382 TRACE neutron.agent.l3.router_info Stderr: Cannot open network namespace "qrouter-8fc0f640-35bb-4d0b-bbbd-80c22be0e762": No such file or directory 2016-04-21 16:22:17.771 23382 TRACE neutron.agent.l3.router_info 2016-04-21 16:22:17.771 23382 TRACE neutron.agent.l3.router_info 2016-04-21 16:22:17.774 23382 ERROR neutron.agent.l3.agent [-] Failed to process compatible router '8fc0f640-35bb-4d0b-bbbd-80c22be0e762' 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent Traceback (most recent call last): 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent File "/usr/lib/python2.7/dist-packages/neutron/agent/l3/agent.py", line 467, in _process_router_update 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent self._process_router_if_compatible(router) 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent File "/usr/lib/python2.7/dist-packages/neutron/agent/l3/agent.py", line 414, in _process_router_if_compatible 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent self._process_updated_router(router) 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent File "/usr/lib/python2.7/dist-packages/neutron/agent/l3/agent.py", line 428, in _process_updated_router 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent ri.process(self) 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent File "/usr/lib/python2.7/dist-packages/neutron/common/utils.py", line 346, in call 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent self.logger(e) 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 85, in __exit__ 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent six.reraise(self.type_, self.value, self.tb) 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent File "/usr/lib/python2.7/dist-packages/neutron/common/utils.py", line 343, in call 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent return func(*args, **kwargs) 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent File "/usr/lib/python2.7/dist-packages/neutron/agent/l3/router_info.py", line 628, in process 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent self._process_internal_ports() 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent File "/usr/lib/python2.7/dist-packages/neutron/agent/l3/router_info.py", line 404, in _process_internal_ports 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent existing_devices = self._get_existing_devices() 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent File "/usr/lib/python2.7/dist-packages/neutron/agent/l3/router_info.py", line 328, in _get_existing_devices 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent ip_devs = ip_wrapper.get_devices(exclude_loopback=True) 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent File "/usr/lib/python2.7/dist-packages/neutron/agent/linux/ip_lib.py", line 102, in get_devices 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent log_fail_as_error=self.log_fail_as_error 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent File "/usr/lib/python2.7/dist-packages/neutron/agent/linux/utils.py", line 137, in execute 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent raise RuntimeError(m) 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent RuntimeError: 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-8fc0f640-35bb-4d0b-bbbd-80c22be0e762', 'find', '/sys/class/net', '-maxdepth', '1', '-type', 'l', '-printf', '%f '] 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent Exit code: 1 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent Stdin: 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent Stdout: 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent Stderr: Cannot open network namespace "qrouter-8fc0f640-35bb-4d0b-bbbd-80c22be0e762": No such file or directory 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent ** Affects: neutron Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1573073 Title: When router has no ports _process_updated_router fails because the namespace does not exist Status in neutron: New Bug description: Happens in Kilo. Cannot test on other releases. Steps to reproduce, create a router and dont add any port. The command 'neutron router-port-list' should return an empty set. As soon as you create the router, the namespace qrouter-<uuid> is present on the network node, but after a while it is purged. As soon as the namespace is not there anymore check the log file vpn-agent.log for the stacktrace. When a router has no ports the namespace is deleted from the network node. However this brakes the router updates and the file vpn- agent.log is flooded with this traces: 2016-04-21 16:22:17.771 23382 TRACE neutron.agent.l3.router_info Traceback (most recent call last): 2016-04-21 16:22:17.771 23382 TRACE neutron.agent.l3.router_info File "/usr/lib/python2.7/dist-packages/neutron/common/utils.py", line 343, in call 2016-04-21 16:22:17.771 23382 TRACE neutron.agent.l3.router_info return func(*args, **kwargs) 2016-04-21 16:22:17.771 23382 TRACE neutron.agent.l3.router_info File "/usr/lib/python2.7/dist-packages/neutron/agent/l3/router_info.py", line 628, in process 2016-04-21 16:22:17.771 23382 TRACE neutron.agent.l3.router_info self._process_internal_ports() 2016-04-21 16:22:17.771 23382 TRACE neutron.agent.l3.router_info File "/usr/lib/python2.7/dist-packages/neutron/agent/l3/router_info.py", line 404, in _process_internal_ports 2016-04-21 16:22:17.771 23382 TRACE neutron.agent.l3.router_info existing_devices = self._get_existing_devices() 2016-04-21 16:22:17.771 23382 TRACE neutron.agent.l3.router_info File "/usr/lib/python2.7/dist-packages/neutron/agent/l3/router_info.py", line 328, in _get_existing_devices 2016-04-21 16:22:17.771 23382 TRACE neutron.agent.l3.router_info ip_devs = ip_wrapper.get_devices(exclude_loopback=True) 2016-04-21 16:22:17.771 23382 TRACE neutron.agent.l3.router_info File "/usr/lib/python2.7/dist-packages/neutron/agent/linux/ip_lib.py", line 102, in get_devices 2016-04-21 16:22:17.771 23382 TRACE neutron.agent.l3.router_info log_fail_as_error=self.log_fail_as_error 2016-04-21 16:22:17.771 23382 TRACE neutron.agent.l3.router_info File "/usr/lib/python2.7/dist-packages/neutron/agent/linux/utils.py", line 137, in execute 2016-04-21 16:22:17.771 23382 TRACE neutron.agent.l3.router_info raise RuntimeError(m) 2016-04-21 16:22:17.771 23382 TRACE neutron.agent.l3.router_info RuntimeError: 2016-04-21 16:22:17.771 23382 TRACE neutron.agent.l3.router_info Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-8fc0f640-35bb-4d0b-bbbd-80c22be0e762', 'find', '/sys/class/net', '-maxdepth', '1', '-type', 'l', '-printf', '%f '] 2016-04-21 16:22:17.771 23382 TRACE neutron.agent.l3.router_info Exit code: 1 2016-04-21 16:22:17.771 23382 TRACE neutron.agent.l3.router_info Stdin: 2016-04-21 16:22:17.771 23382 TRACE neutron.agent.l3.router_info Stdout: 2016-04-21 16:22:17.771 23382 TRACE neutron.agent.l3.router_info Stderr: Cannot open network namespace "qrouter-8fc0f640-35bb-4d0b-bbbd-80c22be0e762": No such file or directory 2016-04-21 16:22:17.771 23382 TRACE neutron.agent.l3.router_info 2016-04-21 16:22:17.771 23382 TRACE neutron.agent.l3.router_info 2016-04-21 16:22:17.774 23382 ERROR neutron.agent.l3.agent [-] Failed to process compatible router '8fc0f640-35bb-4d0b-bbbd-80c22be0e762' 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent Traceback (most recent call last): 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent File "/usr/lib/python2.7/dist-packages/neutron/agent/l3/agent.py", line 467, in _process_router_update 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent self._process_router_if_compatible(router) 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent File "/usr/lib/python2.7/dist-packages/neutron/agent/l3/agent.py", line 414, in _process_router_if_compatible 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent self._process_updated_router(router) 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent File "/usr/lib/python2.7/dist-packages/neutron/agent/l3/agent.py", line 428, in _process_updated_router 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent ri.process(self) 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent File "/usr/lib/python2.7/dist-packages/neutron/common/utils.py", line 346, in call 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent self.logger(e) 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 85, in __exit__ 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent six.reraise(self.type_, self.value, self.tb) 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent File "/usr/lib/python2.7/dist-packages/neutron/common/utils.py", line 343, in call 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent return func(*args, **kwargs) 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent File "/usr/lib/python2.7/dist-packages/neutron/agent/l3/router_info.py", line 628, in process 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent self._process_internal_ports() 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent File "/usr/lib/python2.7/dist-packages/neutron/agent/l3/router_info.py", line 404, in _process_internal_ports 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent existing_devices = self._get_existing_devices() 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent File "/usr/lib/python2.7/dist-packages/neutron/agent/l3/router_info.py", line 328, in _get_existing_devices 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent ip_devs = ip_wrapper.get_devices(exclude_loopback=True) 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent File "/usr/lib/python2.7/dist-packages/neutron/agent/linux/ip_lib.py", line 102, in get_devices 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent log_fail_as_error=self.log_fail_as_error 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent File "/usr/lib/python2.7/dist-packages/neutron/agent/linux/utils.py", line 137, in execute 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent raise RuntimeError(m) 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent RuntimeError: 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-8fc0f640-35bb-4d0b-bbbd-80c22be0e762', 'find', '/sys/class/net', '-maxdepth', '1', '-type', 'l', '-printf', '%f '] 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent Exit code: 1 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent Stdin: 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent Stdout: 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent Stderr: Cannot open network namespace "qrouter-8fc0f640-35bb-4d0b-bbbd-80c22be0e762": No such file or directory 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent 2016-04-21 16:22:17.774 23382 TRACE neutron.agent.l3.agent To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1573073/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp