** Project changed: ovsdbapp => neutron
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1894136
Title:
[OVN Octavia Provider] OVN provider fails during listener delete
Status in neutron:
In Progress
Bug description:
The OVN provider is consistently failing during a listener delete as
part of the member API tempest test tear down with a 'filedescriptor
out of range in select()' error.
o-api logs snippet:
Sep 03 15:44:05.171297 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: ERROR ovn_octavia_provider.ovsdb.impl_idl_ovn
[None req-9201aee8-9a5b-460c-bf8b-c6408d20aec7 tempest-MemberAPITest-903346660
tempest-MemberAPITest-903346660] OVS database connection to OVN_Northbound
failed with error: 'filedescriptor out of range in select()'. Verify that the
OVS and OVN services are available and that the 'ovn_nb_connection' and
'ovn_sb_connection' configuration options are correct.: ValueError:
filedescriptor out of range in select()
Sep 03 15:44:05.171297 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: ERROR ovn_octavia_provider.ovsdb.impl_idl_ovn
Traceback (most recent call last):
Sep 03 15:44:05.171297 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: ERROR ovn_octavia_provider.ovsdb.impl_idl_ovn
File
"/opt/stack/ovn-octavia-provider/ovn_octavia_provider/ovsdb/impl_idl_ovn.py",
line 61, in start_connection
Sep 03 15:44:05.171297 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: ERROR ovn_octavia_provider.ovsdb.impl_idl_ovn
self.ovsdb_connection.start()
Sep 03 15:44:05.171297 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: ERROR ovn_octavia_provider.ovsdb.impl_idl_ovn
File
"/usr/local/lib/python3.6/dist-packages/ovsdbapp/backend/ovs_idl/connection.py",
line 79, in start
Sep 03 15:44:05.171297 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: ERROR ovn_octavia_provider.ovsdb.impl_idl_ovn
idlutils.wait_for_change(self.idl, self.timeout)
Sep 03 15:44:05.171297 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: ERROR ovn_octavia_provider.ovsdb.impl_idl_ovn
File
"/usr/local/lib/python3.6/dist-packages/ovsdbapp/backend/ovs_idl/idlutils.py",
line 201, in wait_for_change
Sep 03 15:44:05.171297 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: ERROR ovn_octavia_provider.ovsdb.impl_idl_ovn
ovs_poller.block()
Sep 03 15:44:05.171297 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: ERROR ovn_octavia_provider.ovsdb.impl_idl_ovn
File "/usr/local/lib/python3.6/dist-packages/ovs/poller.py", line 231, in block
Sep 03 15:44:05.171297 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: ERROR ovn_octavia_provider.ovsdb.impl_idl_ovn
events = self.poll.poll(self.timeout)
Sep 03 15:44:05.171297 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: ERROR ovn_octavia_provider.ovsdb.impl_idl_ovn
File "/usr/local/lib/python3.6/dist-packages/ovs/poller.py", line 140, in poll
Sep 03 15:44:05.171297 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: ERROR ovn_octavia_provider.ovsdb.impl_idl_ovn
timeout)
Sep 03 15:44:05.171297 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: ERROR ovn_octavia_provider.ovsdb.impl_idl_ovn
ValueError: filedescriptor out of range in select()
Sep 03 15:44:05.171297 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: ERROR ovn_octavia_provider.ovsdb.impl_idl_ovn
Sep 03 15:44:05.172746 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: ERROR octavia.api.drivers.driver_factory [None
req-9201aee8-9a5b-460c-bf8b-c6408d20aec7 tempest-MemberAPITest-903346660
tempest-MemberAPITest-903346660] Unable to load provider driver ovn due to: OVS
database connection to OVN_Northbound failed with error: 'filedescriptor out of
range in select()'. Verify that the OVS and OVN services are available and that
the 'ovn_nb_connection' and 'ovn_sb_connection' configuration options are
correct.: ovn_octavia_provider.ovsdb.impl_idl_ovn.OvsdbConnectionUnavailable:
OVS database connection to OVN_Northbound failed with error: 'filedescriptor
out of range in select()'. Verify that the OVS and OVN services are available
and that the 'ovn_nb_connection' and 'ovn_sb_connection' configuration options
are correct.
Sep 03 15:44:05.175074 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: ERROR wsme.api [None
req-9201aee8-9a5b-460c-bf8b-c6408d20aec7 tempest-MemberAPITest-903346660
tempest-MemberAPITest-903346660] Server-side error: "Provider 'ovn' was not
found.". Detail:
Sep 03 15:44:05.175074 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: Traceback (most recent call last):
Sep 03 15:44:05.175074 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: File
"/opt/stack/ovn-octavia-provider/ovn_octavia_provider/ovsdb/impl_idl_ovn.py",
line 61, in start_connection
Sep 03 15:44:05.175074 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: self.ovsdb_connection.start()
Sep 03 15:44:05.175074 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: File
"/usr/local/lib/python3.6/dist-packages/ovsdbapp/backend/ovs_idl/connection.py",
line 79, in start
Sep 03 15:44:05.175074 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: idlutils.wait_for_change(self.idl,
self.timeout)
Sep 03 15:44:05.175074 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: File
"/usr/local/lib/python3.6/dist-packages/ovsdbapp/backend/ovs_idl/idlutils.py",
line 201, in wait_for_change
Sep 03 15:44:05.175074 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: ovs_poller.block()
Sep 03 15:44:05.175074 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: File
"/usr/local/lib/python3.6/dist-packages/ovs/poller.py", line 231, in block
Sep 03 15:44:05.175074 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: events = self.poll.poll(self.timeout)
Sep 03 15:44:05.175074 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: File
"/usr/local/lib/python3.6/dist-packages/ovs/poller.py", line 140, in poll
Sep 03 15:44:05.175074 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: timeout)
Sep 03 15:44:05.175074 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: ValueError: filedescriptor out of range in
select()
Sep 03 15:44:05.175074 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: During handling of the above exception, another
exception occurred:
Sep 03 15:44:05.175074 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: Traceback (most recent call last):
Sep 03 15:44:05.175074 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: File
"/opt/stack/octavia/octavia/api/drivers/driver_factory.py", line 44, in
get_driver
Sep 03 15:44:05.175074 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: invoke_on_load=True).driver
Sep 03 15:44:05.175074 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: File
"/usr/local/lib/python3.6/dist-packages/stevedore/driver.py", line 61, in
__init__
Sep 03 15:44:05.175074 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]:
warn_on_missing_entrypoint=warn_on_missing_entrypoint
Sep 03 15:44:05.175074 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: File
"/usr/local/lib/python3.6/dist-packages/stevedore/named.py", line 81, in
__init__
Sep 03 15:44:05.175074 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: verify_requirements)
Sep 03 15:44:05.175074 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: File
"/usr/local/lib/python3.6/dist-packages/stevedore/extension.py", line 233, in
_load_plugins
Sep 03 15:44:05.175074 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: self._on_load_failure_callback(self, ep, err)
Sep 03 15:44:05.175074 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: File
"/usr/local/lib/python3.6/dist-packages/stevedore/extension.py", line 225, in
_load_plugins
Sep 03 15:44:05.175074 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: verify_requirements,
Sep 03 15:44:05.175074 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: File
"/usr/local/lib/python3.6/dist-packages/stevedore/named.py", line 158, in
_load_one_plugin
Sep 03 15:44:05.175074 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: verify_requirements,
Sep 03 15:44:05.175074 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: File
"/usr/local/lib/python3.6/dist-packages/stevedore/extension.py", line 257, in
_load_one_plugin
Sep 03 15:44:05.175074 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: obj = plugin(*invoke_args, **invoke_kwds)
Sep 03 15:44:05.175074 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: File
"/opt/stack/ovn-octavia-provider/ovn_octavia_provider/driver.py", line 42, in
__init__
Sep 03 15:44:05.175074 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: self._ovn_helper =
ovn_helper.OvnProviderHelper()
Sep 03 15:44:05.175074 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: File
"/opt/stack/ovn-octavia-provider/ovn_octavia_provider/helper.py", line 60, in
__init__
Sep 03 15:44:05.175074 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: self.ovn_nbdb_api = self.ovn_nbdb.start()
Sep 03 15:44:05.175074 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: File
"/opt/stack/ovn-octavia-provider/ovn_octavia_provider/ovsdb/impl_idl_ovn.py",
line 174, in start
Sep 03 15:44:05.175074 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: return impl_idl_ovn.OvsdbNbOvnIdl(self.conn)
Sep 03 15:44:05.175074 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: File
"/opt/stack/ovn-octavia-provider/ovn_octavia_provider/ovsdb/impl_idl_ovn.py",
line 118, in __init__
Sep 03 15:44:05.175074 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: super(OvsdbNbOvnIdl,
self).__init__(connection)
Sep 03 15:44:05.175074 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: File
"/opt/stack/ovn-octavia-provider/ovn_octavia_provider/ovsdb/impl_idl_ovn.py",
line 57, in __init__
Sep 03 15:44:05.175074 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: super(Backend, self).__init__(connection)
Sep 03 15:44:05.175074 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: File
"/usr/local/lib/python3.6/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py",
line 35, in __init__
Sep 03 15:44:05.175074 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: self.start_connection(connection)
Sep 03 15:44:05.175074 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: File
"/opt/stack/ovn-octavia-provider/ovn_octavia_provider/ovsdb/impl_idl_ovn.py",
line 66, in start_connection
Sep 03 15:44:05.175074 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: raise connection_exception
Sep 03 15:44:05.175074 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]:
ovn_octavia_provider.ovsdb.impl_idl_ovn.OvsdbConnectionUnavailable: OVS
database connection to OVN_Northbound failed with error: 'filedescriptor out of
range in select()'. Verify that the OVS and OVN services are available and that
the 'ovn_nb_connection' and 'ovn_sb_connection' configuration options are
correct.
Sep 03 15:44:05.175074 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: During handling of the above exception, another
exception occurred:
Sep 03 15:44:05.175074 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: Traceback (most recent call last):
Sep 03 15:44:05.175074 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: File
"/usr/local/lib/python3.6/dist-packages/wsmeext/pecan.py", line 84, in
callfunction
Sep 03 15:44:05.175074 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: result = f(self, *args, **kwargs)
Sep 03 15:44:05.175074 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: File
"/opt/stack/octavia/octavia/api/v2/controllers/listener.py", line 632, in delete
Sep 03 15:44:05.175074 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: driver = driver_factory.get_driver(provider)
Sep 03 15:44:05.175074 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: File
"/opt/stack/octavia/octavia/api/drivers/driver_factory.py", line 49, in
get_driver
Sep 03 15:44:05.175074 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: raise
exceptions.ProviderNotFound(prov=provider)
Sep 03 15:44:05.175074 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: octavia.common.exceptions.ProviderNotFound:
Provider 'ovn' was not found.
Sep 03 15:44:05.175074 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: : octavia.common.exceptions.ProviderNotFound:
Provider 'ovn' was not found.
Sep 03 15:44:05.178879 ubuntu-bionic-inap-mtl01-0019624710
devstack@o-api.service[10087]: [pid: 10089|app: 0|req: 1109/2281] 198.72.124.16
() {56 vars in 1234 bytes} [Thu Sep 3 15:44:04 2020] DELETE
/load-balancer/v2.0/lbaas/listeners/3537f411-5ab3-48c8-9669-c9fa1f40fee9 =>
generated 90 bytes in 447 msecs (HTTP/1.1 501) 4 headers in 169 bytes (1
switches on core 0)
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1894136/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
