** Also affects: neutron/kilo Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1448148
Title: Device not found: exceptions in l3 grenade Status in neutron: Fix Released Status in neutron kilo series: New Bug description: exceptions have been seen in screen-q-vpn.txt.gz at the gate, and it happens quite often recently: 2015-04-23 15:55:56.236 ERROR neutron.agent.l3.agent [req-5ea4d9d1-66ab-444c-a66f-c48094f3582d None None] Failed to process compatible router 'aeb00076-1c9e-431d-973f-ce1123c918a7' 2015-04-23 15:55:56.236 3736 TRACE neutron.agent.l3.agent Traceback (most recent call last): 2015-04-23 15:55:56.236 3736 TRACE neutron.agent.l3.agent File "/opt/stack/old/neutron/neutron/agent/l3/agent.py", line 452, in _process_router_update 2015-04-23 15:55:56.236 3736 TRACE neutron.agent.l3.agent self._process_router_if_compatible(router) 2015-04-23 15:55:56.236 3736 TRACE neutron.agent.l3.agent File "/opt/stack/old/neutron/neutron/agent/l3/agent.py", line 404, in _process_router_if_compatible 2015-04-23 15:55:56.236 3736 TRACE neutron.agent.l3.agent self._process_added_router(router) 2015-04-23 15:55:56.236 3736 TRACE neutron.agent.l3.agent File "/opt/stack/old/neutron/neutron/agent/l3/agent.py", line 412, in _process_added_router 2015-04-23 15:55:56.236 3736 TRACE neutron.agent.l3.agent ri.process(self) 2015-04-23 15:55:56.236 3736 TRACE neutron.agent.l3.agent File "/opt/stack/old/neutron/neutron/common/utils.py", line 346, in call 2015-04-23 15:55:56.236 3736 TRACE neutron.agent.l3.agent self.logger(e) 2015-04-23 15:55:56.236 3736 TRACE neutron.agent.l3.agent File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 85, in __exit__ 2015-04-23 15:55:56.236 3736 TRACE neutron.agent.l3.agent six.reraise(self.type_, self.value, self.tb) 2015-04-23 15:55:56.236 3736 TRACE neutron.agent.l3.agent File "/opt/stack/old/neutron/neutron/common/utils.py", line 343, in call 2015-04-23 15:55:56.236 3736 TRACE neutron.agent.l3.agent return func(*args, **kwargs) 2015-04-23 15:55:56.236 3736 TRACE neutron.agent.l3.agent File "/opt/stack/old/neutron/neutron/agent/l3/router_info.py", line 605, in process 2015-04-23 15:55:56.236 3736 TRACE neutron.agent.l3.agent self._process_internal_ports() 2015-04-23 15:55:56.236 3736 TRACE neutron.agent.l3.agent File "/opt/stack/old/neutron/neutron/agent/l3/router_info.py", line 361, in _process_internal_ports 2015-04-23 15:55:56.236 3736 TRACE neutron.agent.l3.agent self.internal_network_added(p) 2015-04-23 15:55:56.236 3736 TRACE neutron.agent.l3.agent File "/opt/stack/old/neutron/neutron/agent/l3/router_info.py", line 312, in internal_network_added 2015-04-23 15:55:56.236 3736 TRACE neutron.agent.l3.agent INTERNAL_DEV_PREFIX) 2015-04-23 15:55:56.236 3736 TRACE neutron.agent.l3.agent File "/opt/stack/old/neutron/neutron/agent/l3/router_info.py", line 288, in _internal_network_added 2015-04-23 15:55:56.236 3736 TRACE neutron.agent.l3.agent prefix=prefix) 2015-04-23 15:55:56.236 3736 TRACE neutron.agent.l3.agent File "/opt/stack/old/neutron/neutron/agent/linux/interface.py", line 264, in plug 2015-04-23 15:55:56.236 3736 TRACE neutron.agent.l3.agent ns_dev.link.set_up() 2015-04-23 15:55:56.236 3736 TRACE neutron.agent.l3.agent File "/opt/stack/old/neutron/neutron/agent/linux/ip_lib.py", line 276, in set_up 2015-04-23 15:55:56.236 3736 TRACE neutron.agent.l3.agent self._as_root([], ('set', self.name, 'up')) 2015-04-23 15:55:56.236 3736 TRACE neutron.agent.l3.agent File "/opt/stack/old/neutron/neutron/agent/linux/ip_lib.py", line 222, in _as_root 2015-04-23 15:55:56.236 3736 TRACE neutron.agent.l3.agent use_root_namespace=use_root_namespace) 2015-04-23 15:55:56.236 3736 TRACE neutron.agent.l3.agent File "/opt/stack/old/neutron/neutron/agent/linux/ip_lib.py", line 69, in _as_root 2015-04-23 15:55:56.236 3736 TRACE neutron.agent.l3.agent log_fail_as_error=self.log_fail_as_error) 2015-04-23 15:55:56.236 3736 TRACE neutron.agent.l3.agent File "/opt/stack/old/neutron/neutron/agent/linux/ip_lib.py", line 78, in _execute 2015-04-23 15:55:56.236 3736 TRACE neutron.agent.l3.agent log_fail_as_error=log_fail_as_error) 2015-04-23 15:55:56.236 3736 TRACE neutron.agent.l3.agent File "/opt/stack/old/neutron/neutron/agent/linux/utils.py", line 137, in execute 2015-04-23 15:55:56.236 3736 TRACE neutron.agent.l3.agent raise RuntimeError(m) 2015-04-23 15:55:56.236 3736 TRACE neutron.agent.l3.agent RuntimeError: 2015-04-23 15:55:56.236 3736 TRACE neutron.agent.l3.agent Command: ['ip', 'netns', 'exec', u'qrouter-aeb00076-1c9e-431d-973f-ce1123c918a7', 'ip', 'link', 'set', u'qr-91aaad43-1e', 'up'] 2015-04-23 15:55:56.236 3736 TRACE neutron.agent.l3.agent Exit code: 1 2015-04-23 15:55:56.236 3736 TRACE neutron.agent.l3.agent Stdin: 2015-04-23 15:55:56.236 3736 TRACE neutron.agent.l3.agent Stdout: 2015-04-23 15:55:56.236 3736 TRACE neutron.agent.l3.agent Stderr: Cannot find device "qr-91aaad43-1e" Do a search at http://logstash.openstack.org/#eyJzZWFyY2giOiJtZXNzYWdlOiBcIkNhbm5vdCBmaW5kIGRldmljZVwiIEFORCBtZXNzYWdlOiBcInFyLVwiIiwiZmllbGRzIjpbXSwib2Zmc2V0IjowLCJ0aW1lZnJhbWUiOiI2MDQ4MDAiLCJncmFwaG1vZGUiOiJjb3VudCIsInRpbWUiOnsidXNlcl9pbnRlcnZhbCI6MH0sInN0YW1wIjoxNDI5ODgzMDUxOTIxfQ== With the search string: message: "Cannot find device" AND message: "qr-", many occurrences have been seen. This seems to be the reason for failures of check-grenade-dvm-neutron as seen in this review: https://review.openstack.org/#/c/176041/ although not of the test cases failed. To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1448148/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp