** Also affects: neutron (Ubuntu)
Importance: Undecided
Status: New
** Also affects: neutron (Ubuntu Bionic)
Importance: Undecided
Status: New
** Changed in: neutron (Ubuntu)
Status: New => Fix Released
** Changed in: neutron (Ubuntu Bionic)
Assignee: (unassigned) => Seyeong Kim (seyeongkim)
** Changed in: neutron (Ubuntu Bionic)
Status: New => In Progress
** Patch added: "lp1849098_bionic.debdiff"
https://bugs.launchpad.net/ubuntu/+source/neutron/+bug/1849098/+attachment/5494377/+files/lp1849098_bionic.debdiff
** Description changed:
+ [Impact]
+
+ somehow port is unbounded, then neutron-openvswitch-agent raise
+ OVSFWTagNotFound, then creating new instance will be failed.
+
+ [Test Plan]
+ 1. deploy bionic openstack env
+ 2. launch one instance
+ 3. modify neutron-openvswitch-agent code inside nova-compute
+ - https://pastebin.ubuntu.com/p/nBRKkXmjx8/
+ 4. restart neutron-openvswitch-agent
+ 5. check if there are a lot of cannot get tag for port ..
+ 6. launch another instance.
+ 7. It fails after vif_plugging_timeout, with "virtual interface creation
failed"
+
+ [Where problems could occur]
+ You need to restart service. and as patch, Basically it will be ok as it adds
only exceptions. but getting or creating vif port part can have issue.
+
+ [Others]
+
+ Original description.
+
neutron-openvswitch-agent meets unbound port:
2019-10-17 11:32:21.868 135 WARNING
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-
aae68b42-a99f-4bb3-bcf6-a6d3c4ca9e31 - - - - -] Device
ef34215f-e099-4fd0-935f-c9a42951d166 not defined on plugin or binding
failed
Later when applying firewall rules:
2019-10-17 11:32:21.901 135 INFO neutron.agent.securitygroups_rpc
[req-aae68b42-a99f-4bb3-bcf6-a6d3c4ca9e31 - - - - -] Preparing filters for
devices {'ef34215f-e099-4fd0-935f-c9a42951d166',
'e9c97cf0-1a5e-4d77-b57b-0ba474d12e29', 'fff1bb24-6423-4486-87c4-1fe17c552cca',
'2e20f9ee-bcb5-445c-b31f-d70d276d45c9', '03a60047-cb07-42a4-8b49-619d5982a9bd',
'a452cea2-deaf-4411-bbae-ce83870cbad4', '79b03e5c-9be0-4808-9784-cb4878c3dbd5',
'9b971e75-3c1b-463d-88cf-3f298105fa6e'}
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
[req-aae68b42-a99f-4bb3-bcf6-a6d3c4ca9e31 - - - - -] Error while processing VIF
ports: neutron.agent.linux.openvswitch_firewall.exceptions.OVSFWTagNotFound:
Cannot get tag for port o-hm0 from its other_config: {}
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent Traceback (most
recent call last):
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/var/lib/openstack/lib/python3.6/site-packages/neutron/agent/linux/openvswitch_firewall/firewall.py",
line 530, in get_or_create_ofport
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent of_port =
self.sg_port_map.ports[port_id]
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent KeyError:
'ef34215f-e099-4fd0-935f-c9a42951d166'
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent During handling
of the above exception, another exception occurred:
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent Traceback (most
recent call last):
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/var/lib/openstack/lib/python3.6/site-packages/neutron/agent/linux/openvswitch_firewall/firewall.py",
line 81, in get_tag_from_other_config
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent return
int(other_config['tag'])
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent KeyError: 'tag'
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent During handling
of the above exception, another exception occurred:
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent Traceback (most
recent call last):
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/var/lib/openstack/lib/python3.6/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py",
line 2280, in rpc_loop
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent port_info,
provisioning_needed)
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/var/lib/openstack/lib/python3.6/site-packages/osprofiler/profiler.py", line
160, in wrapper
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent result =
f(*args, **kwargs)
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/var/lib/openstack/lib/python3.6/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py",
line 1847, in process_network_ports
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
port_info.get('updated', set()))
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/var/lib/openstack/lib/python3.6/site-packages/neutron/agent/securitygroups_rpc.py",
line 258, in setup_port_filters
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
self.prepare_devices_filter(new_devices)
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/var/lib/openstack/lib/python3.6/site-packages/neutron/agent/securitygroups_rpc.py",
line 125, in decorated_function
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent *args,
**kwargs)
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/var/lib/openstack/lib/python3.6/site-packages/neutron/agent/securitygroups_rpc.py",
line 133, in prepare_devices_filter
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
self._apply_port_filter(device_ids)
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/var/lib/openstack/lib/python3.6/site-packages/neutron/agent/securitygroups_rpc.py",
line 164, in _apply_port_filter
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
self.firewall.prepare_port_filter(device)
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/var/lib/openstack/lib/python3.6/site-packages/neutron/agent/linux/openvswitch_firewall/firewall.py",
line 555, in prepare_port_filter
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent of_port =
self.get_or_create_ofport(port)
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/var/lib/openstack/lib/python3.6/site-packages/neutron/agent/linux/openvswitch_firewall/firewall.py",
line 532, in get_or_create_ofport
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
port_vlan_id = self._get_port_vlan_tag(ovs_port.port_name)
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/var/lib/openstack/lib/python3.6/site-packages/neutron/agent/linux/openvswitch_firewall/firewall.py",
line 516, in _get_port_vlan_tag
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent return
get_tag_from_other_config(self.int_br.br, port_name)
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/var/lib/openstack/lib/python3.6/site-packages/neutron/agent/linux/openvswitch_firewall/firewall.py",
line 84, in get_tag_from_other_config
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
port_name=port_name, other_config=other_config)
2019-10-17 11:32:21.906 135 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
neutron.agent.linux.openvswitch_firewall.exceptions.OVSFWTagNotFound: Cannot
get tag for port o-hm0 from its other_config: {}
2019-10-17 11:32:21.909 135 INFO
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
[req-aae68b42-a99f-4bb3-bcf6-a6d3c4ca9e31 - - - - -] Agent out of sync with
plugin!
this happens in each agent cycle so agent can't do anything.
Need to handle OVSFWTagNotFound in prepare_port_filter() like was done
for update_port_filter in https://review.opendev.org/#/c/630910/
** Tags added: sts
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1849098
Title:
ovs agent is stuck with OVSFWTagNotFound when dealing with unbound
port
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1849098/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
