Public bug reported: Steps to reproduce:
Single node devstack setup. Start devstack with ML2 and ovs,l2 population drivers. Enable DVR in L3/L2 and Service plugin (with DVR_SNAT) 1. Create Network with an instance on it 2. Create Router attached to external network 3. Add the network to the router (Two new interfaces should be added to the router, the network and an SNAT interface 4. Delete the two added interfaces from the router (or just the network one) The following exception is shown in the L3 Agent: 2015-03-22 16:07:54.613 ERROR neutron.agent.linux.utils [-] Command: ['sudo', '/usr/bin/neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-81176deb-cc2b-4d29-903e-0a1a2fe207d6', 'ip', '-4', 'route', 'del', 'default', 'via', '10.1.0.8', 'dev', 'qr-a3c4575d-5f', 'table', '167837697'] Exit code: 1 Stdin: Stdout: Stderr: Cannot find device "qr-a3c4575d-5f" 2015-03-22 16:07:54.614 ERROR neutron.agent.l3.dvr_router [-] DVR: removed snat failed 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router Traceback (most recent call last): 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router File "/opt/stack/neutron/neutron/agent/l3/dvr_router.py", line 282, in _snat_redirect_remove 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router ns_ipd.route.delete_gateway(gateway, table=snat_idx) 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router File "/opt/stack/neutron/neutron/agent/linux/ip_lib.py", line 409, in delete_gateway 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router self._as_root([ip_version], tuple(args)) 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router File "/opt/stack/neutron/neutron/agent/linux/ip_lib.py", line 222, in _as_root 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router use_root_namespace=use_root_namespace) 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router File "/opt/stack/neutron/neutron/agent/linux/ip_lib.py", line 69, in _as_root 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router log_fail_as_error=self.log_fail_as_error) 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router File "/opt/stack/neutron/neutron/agent/linux/ip_lib.py", line 78, in _execute 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router log_fail_as_error=log_fail_as_error) 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router File "/opt/stack/neutron/neutron/agent/linux/utils.py", line 135, in execute 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router raise RuntimeError(m) 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router RuntimeError: 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router Command: ['sudo', '/usr/bin/neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-81176deb-cc2b-4d29-903e-0a1a2fe207d6', 'ip', '-4', 'route', 'del', 'default', 'via', '10.1.0.8', 'dev', 'qr-a3c4575d-5f', 'table', '167837697'] 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router Exit code: 1 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router Stdin: 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router Stdout: 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router Stderr: Cannot find device "qr-a3c4575d-5f" 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router Assumption: The code tries to delete the default route (for the SNAT) inside the router namespace, how ever the namespace is already deleted (since this is only connected network) ** Affects: neutron Importance: Undecided Assignee: Gal Sagie (gal-sagie) Status: New ** Changed in: neutron Assignee: (unassigned) => Gal Sagie (gal-sagie) ** Description changed: Steps to reproduce: Single node devstack setup. Start devstack with ML2 and ovs,l2 population drivers. Enable DVR in L3/L2 and Service plugin (with DVR_SNAT) - 1. Create Network with an instance on it - 2. Create Router attached to external router + 2. Create Router attached to external network 3. Add the network to the router (Two new interfaces should be added to the router, the network and an SNAT interface 4. Delete the two added interfaces from the router (or just the network one) The following exception is shown in the L3 Agent: - 2015-03-22 16:07:54.613 ERROR neutron.agent.linux.utils [-] + 2015-03-22 16:07:54.613 ERROR neutron.agent.linux.utils [-] Command: ['sudo', '/usr/bin/neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-81176deb-cc2b-4d29-903e-0a1a2fe207d6', 'ip', '-4', 'route', 'del', 'default', 'via', '10.1.0.8', 'dev', 'qr-a3c4575d-5f', 'table', '167837697'] Exit code: 1 - Stdin: - Stdout: + Stdin: + Stdout: Stderr: Cannot find device "qr-a3c4575d-5f" 2015-03-22 16:07:54.614 ERROR neutron.agent.l3.dvr_router [-] DVR: removed snat failed 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router Traceback (most recent call last): 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router File "/opt/stack/neutron/neutron/agent/l3/dvr_router.py", line 282, in _snat_redirect_remove 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router ns_ipd.route.delete_gateway(gateway, table=snat_idx) 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router File "/opt/stack/neutron/neutron/agent/linux/ip_lib.py", line 409, in delete_gateway 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router self._as_root([ip_version], tuple(args)) 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router File "/opt/stack/neutron/neutron/agent/linux/ip_lib.py", line 222, in _as_root 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router use_root_namespace=use_root_namespace) 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router File "/opt/stack/neutron/neutron/agent/linux/ip_lib.py", line 69, in _as_root 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router log_fail_as_error=self.log_fail_as_error) 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router File "/opt/stack/neutron/neutron/agent/linux/ip_lib.py", line 78, in _execute 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router log_fail_as_error=log_fail_as_error) 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router File "/opt/stack/neutron/neutron/agent/linux/utils.py", line 135, in execute 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router raise RuntimeError(m) - 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router RuntimeError: + 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router RuntimeError: 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router Command: ['sudo', '/usr/bin/neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-81176deb-cc2b-4d29-903e-0a1a2fe207d6', 'ip', '-4', 'route', 'del', 'default', 'via', '10.1.0.8', 'dev', 'qr-a3c4575d-5f', 'table', '167837697'] 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router Exit code: 1 - 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router Stdin: - 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router Stdout: + 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router Stdin: + 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router Stdout: 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router Stderr: Cannot find device "qr-a3c4575d-5f" - 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router - 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router - + 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router + 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router Assumption: The code tries to delete the default route (for the SNAT) inside the router namespace, how ever the namespace is already deleted (since this is only connected network) -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1435012 Title: Exception when removing last network port from external router Status in OpenStack Neutron (virtual network service): New Bug description: Steps to reproduce: Single node devstack setup. Start devstack with ML2 and ovs,l2 population drivers. Enable DVR in L3/L2 and Service plugin (with DVR_SNAT) 1. Create Network with an instance on it 2. Create Router attached to external network 3. Add the network to the router (Two new interfaces should be added to the router, the network and an SNAT interface 4. Delete the two added interfaces from the router (or just the network one) The following exception is shown in the L3 Agent: 2015-03-22 16:07:54.613 ERROR neutron.agent.linux.utils [-] Command: ['sudo', '/usr/bin/neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-81176deb-cc2b-4d29-903e-0a1a2fe207d6', 'ip', '-4', 'route', 'del', 'default', 'via', '10.1.0.8', 'dev', 'qr-a3c4575d-5f', 'table', '167837697'] Exit code: 1 Stdin: Stdout: Stderr: Cannot find device "qr-a3c4575d-5f" 2015-03-22 16:07:54.614 ERROR neutron.agent.l3.dvr_router [-] DVR: removed snat failed 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router Traceback (most recent call last): 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router File "/opt/stack/neutron/neutron/agent/l3/dvr_router.py", line 282, in _snat_redirect_remove 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router ns_ipd.route.delete_gateway(gateway, table=snat_idx) 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router File "/opt/stack/neutron/neutron/agent/linux/ip_lib.py", line 409, in delete_gateway 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router self._as_root([ip_version], tuple(args)) 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router File "/opt/stack/neutron/neutron/agent/linux/ip_lib.py", line 222, in _as_root 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router use_root_namespace=use_root_namespace) 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router File "/opt/stack/neutron/neutron/agent/linux/ip_lib.py", line 69, in _as_root 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router log_fail_as_error=self.log_fail_as_error) 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router File "/opt/stack/neutron/neutron/agent/linux/ip_lib.py", line 78, in _execute 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router log_fail_as_error=log_fail_as_error) 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router File "/opt/stack/neutron/neutron/agent/linux/utils.py", line 135, in execute 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router raise RuntimeError(m) 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router RuntimeError: 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router Command: ['sudo', '/usr/bin/neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-81176deb-cc2b-4d29-903e-0a1a2fe207d6', 'ip', '-4', 'route', 'del', 'default', 'via', '10.1.0.8', 'dev', 'qr-a3c4575d-5f', 'table', '167837697'] 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router Exit code: 1 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router Stdin: 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router Stdout: 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router Stderr: Cannot find device "qr-a3c4575d-5f" 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router 2015-03-22 16:07:54.614 TRACE neutron.agent.l3.dvr_router Assumption: The code tries to delete the default route (for the SNAT) inside the router namespace, how ever the namespace is already deleted (since this is only connected network) To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1435012/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp