Public bug reported: It happens that tpi patch ports between trunk bridge and integration bridge don't have internal vlan tag in other_config row in ovsdb. It looks like a race between trunk handler and ovs agent.
Example of failure: http://logs.openstack.org/92/527992/2/check/neutron- tempest-plugin-dvr-multinode- scenario/166eee3/logs/subnode-2/screen-q-agt.txt.gz#_Dec_14_18_31_25_801432 Trace example: Dec 14 18:31:25.801432 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [None req-412981e2-ac8a-4fe8-8ec4-288bbb63e2a7 None None] Error while processing VIF ports: OVSFWTagNotFound: Cannot get tag for port tpi-6457d45d-b6 from its other_config: {} Dec 14 18:31:25.801580 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent Traceback (most recent call last): Dec 14 18:31:25.801708 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File "/opt/stack/new/neutron/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py", line 2080, in rpc_loop Dec 14 18:31:25.801838 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent port_info, ovs_restarted) Dec 14 18:31:25.801965 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File "/usr/local/lib/python2.7/dist-packages/osprofiler/profiler.py", line 157, in wrapper Dec 14 18:31:25.802089 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent result = f(*args, **kwargs) Dec 14 18:31:25.802214 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File "/opt/stack/new/neutron/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py", line 1676, in process_network_ports Dec 14 18:31:25.802345 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent port_info.get('updated', set())) Dec 14 18:31:25.802476 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File "/opt/stack/new/neutron/neutron/agent/securitygroups_rpc.py", line 256, in setup_port_filters Dec 14 18:31:25.802600 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent self.refresh_firewall(updated_devices) Dec 14 18:31:25.802725 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File "/opt/stack/new/neutron/neutron/agent/securitygroups_rpc.py", line 110, in decorated_function Dec 14 18:31:25.802850 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent *args, **kwargs) Dec 14 18:31:25.802983 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File "/opt/stack/new/neutron/neutron/agent/securitygroups_rpc.py", line 209, in refresh_firewall Dec 14 18:31:25.803103 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent self._apply_port_filter(device_ids, update_filter=True) Dec 14 18:31:25.803237 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File "/opt/stack/new/neutron/neutron/agent/securitygroups_rpc.py", line 141, in _apply_port_filter Dec 14 18:31:25.803366 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent self.firewall.update_port_filter(device) Dec 14 18:31:25.803492 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File "/opt/stack/new/neutron/neutron/agent/linux/openvswitch_firewall/firewall.py", line 509, in update_port_filter Dec 14 18:31:25.803612 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent self.prepare_port_filter(port) Dec 14 18:31:25.803763 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File "/opt/stack/new/neutron/neutron/agent/linux/openvswitch_firewall/firewall.py", line 492, in prepare_port_filter Dec 14 18:31:25.803894 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent of_port = self.get_or_create_ofport(port) Dec 14 18:31:25.804021 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File "/opt/stack/new/neutron/neutron/agent/linux/openvswitch_firewall/firewall.py", line 464, in get_or_create_ofport Dec 14 18:31:25.804152 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent port_vlan_id = self._get_port_vlan_tag(ovs_port.port_name) Dec 14 18:31:25.804275 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File "/opt/stack/new/neutron/neutron/agent/linux/openvswitch_firewall/firewall.py", line 448, in _get_port_vlan_tag Dec 14 18:31:25.804414 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent return get_tag_from_other_config(self.int_br.br, port_name) Dec 14 18:31:25.804548 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File "/opt/stack/new/neutron/neutron/agent/linux/openvswitch_firewall/firewall.py", line 74, in get_tag_from_other_config Dec 14 18:31:25.804670 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent port_name=port_name, other_config=other_config) Dec 14 18:31:25.804794 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent OVSFWTagNotFound: Cannot get tag for port tpi-6457d45d-b6 from its other_config: {} Dec 14 18:31:25.804923 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent ** Affects: neutron Importance: High Assignee: Jakub Libosvar (libosvar) Status: Confirmed ** Tags: ovs-fw sg-fw trunk -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1740885 Title: Trunk ports are sometimes not tagged with internal vlan Status in neutron: Confirmed Bug description: It happens that tpi patch ports between trunk bridge and integration bridge don't have internal vlan tag in other_config row in ovsdb. It looks like a race between trunk handler and ovs agent. Example of failure: http://logs.openstack.org/92/527992/2/check /neutron-tempest-plugin-dvr-multinode- scenario/166eee3/logs/subnode-2/screen-q-agt.txt.gz#_Dec_14_18_31_25_801432 Trace example: Dec 14 18:31:25.801432 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [None req-412981e2-ac8a-4fe8-8ec4-288bbb63e2a7 None None] Error while processing VIF ports: OVSFWTagNotFound: Cannot get tag for port tpi-6457d45d-b6 from its other_config: {} Dec 14 18:31:25.801580 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent Traceback (most recent call last): Dec 14 18:31:25.801708 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File "/opt/stack/new/neutron/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py", line 2080, in rpc_loop Dec 14 18:31:25.801838 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent port_info, ovs_restarted) Dec 14 18:31:25.801965 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File "/usr/local/lib/python2.7/dist-packages/osprofiler/profiler.py", line 157, in wrapper Dec 14 18:31:25.802089 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent result = f(*args, **kwargs) Dec 14 18:31:25.802214 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File "/opt/stack/new/neutron/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py", line 1676, in process_network_ports Dec 14 18:31:25.802345 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent port_info.get('updated', set())) Dec 14 18:31:25.802476 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File "/opt/stack/new/neutron/neutron/agent/securitygroups_rpc.py", line 256, in setup_port_filters Dec 14 18:31:25.802600 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent self.refresh_firewall(updated_devices) Dec 14 18:31:25.802725 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File "/opt/stack/new/neutron/neutron/agent/securitygroups_rpc.py", line 110, in decorated_function Dec 14 18:31:25.802850 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent *args, **kwargs) Dec 14 18:31:25.802983 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File "/opt/stack/new/neutron/neutron/agent/securitygroups_rpc.py", line 209, in refresh_firewall Dec 14 18:31:25.803103 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent self._apply_port_filter(device_ids, update_filter=True) Dec 14 18:31:25.803237 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File "/opt/stack/new/neutron/neutron/agent/securitygroups_rpc.py", line 141, in _apply_port_filter Dec 14 18:31:25.803366 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent self.firewall.update_port_filter(device) Dec 14 18:31:25.803492 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File "/opt/stack/new/neutron/neutron/agent/linux/openvswitch_firewall/firewall.py", line 509, in update_port_filter Dec 14 18:31:25.803612 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent self.prepare_port_filter(port) Dec 14 18:31:25.803763 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File "/opt/stack/new/neutron/neutron/agent/linux/openvswitch_firewall/firewall.py", line 492, in prepare_port_filter Dec 14 18:31:25.803894 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent of_port = self.get_or_create_ofport(port) Dec 14 18:31:25.804021 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File "/opt/stack/new/neutron/neutron/agent/linux/openvswitch_firewall/firewall.py", line 464, in get_or_create_ofport Dec 14 18:31:25.804152 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent port_vlan_id = self._get_port_vlan_tag(ovs_port.port_name) Dec 14 18:31:25.804275 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File "/opt/stack/new/neutron/neutron/agent/linux/openvswitch_firewall/firewall.py", line 448, in _get_port_vlan_tag Dec 14 18:31:25.804414 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent return get_tag_from_other_config(self.int_br.br, port_name) Dec 14 18:31:25.804548 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File "/opt/stack/new/neutron/neutron/agent/linux/openvswitch_firewall/firewall.py", line 74, in get_tag_from_other_config Dec 14 18:31:25.804670 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent port_name=port_name, other_config=other_config) Dec 14 18:31:25.804794 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent OVSFWTagNotFound: Cannot get tag for port tpi-6457d45d-b6 from its other_config: {} Dec 14 18:31:25.804923 ubuntu-xenial-rax-ord-0001444835 neutron-openvswitch-agent[17015]: ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1740885/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
