[Yahoo-eng-team] [Bug 2032770] Re: [OVN] port creation with --enable-uplink-status-propagation does not work with OVN mechanism driver
** Also affects: cloud-archive/victoria Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/2032770 Title: [OVN] port creation with --enable-uplink-status-propagation does not work with OVN mechanism driver Status in Ubuntu Cloud Archive: New Status in Ubuntu Cloud Archive antelope series: New Status in Ubuntu Cloud Archive ussuri series: Fix Committed Status in Ubuntu Cloud Archive victoria series: New Status in Ubuntu Cloud Archive wallaby series: New Status in Ubuntu Cloud Archive xena series: New Status in Ubuntu Cloud Archive yoga series: New Status in Ubuntu Cloud Archive zed series: New Status in neutron: Fix Released Status in neutron package in Ubuntu: New Status in neutron source package in Focal: New Status in neutron source package in Jammy: New Status in neutron source package in Lunar: New Bug description: The port "uplink_status_propagation" feature does not work when OVN is used as the mechanism driver. The reproducer below is working fine with openvswitch as the mechanism driver, but not with the OVN: openstack port create --binding-profile trusted=true --enable-uplink- status-propagation --net private --vnic-type direct test-sriov-bond- enable-uplink-status-propagation-vm-1-port-1 The command fails with the following error when OVN is the mech driver: BadRequestException: 400: Client Error for url: https://10.5.3.81:9696/v2.0/ports, Unrecognized attribute(s) 'propagate_uplink_status' With ML2/OVS, the port creation command above succeeds without any errors. As for the ml2_conf, "uplink_status_propagation" is listed in the extension drivers: [ml2] extension_drivers=port_security,dns_domain_ports,uplink_status_propagation type_drivers = geneve,gre,vlan,flat,local tenant_network_types = geneve,gre,vlan,flat,local mechanism_drivers = ovn,sriovnicswitch /*...*/ I also found the following document which shows the feature gap between ML2/OVS and OVN, but the uplink_status_propagation is not listed: https://docs.openstack.org/neutron/latest/ovn/gaps.html#id9 , maybe this page can be updated as well. To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/2032770/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 2032770] Re: [OVN] port creation with --enable-uplink-status-propagation does not work with OVN mechanism driver
** Also affects: neutron (Ubuntu Focal) Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/2032770 Title: [OVN] port creation with --enable-uplink-status-propagation does not work with OVN mechanism driver Status in Ubuntu Cloud Archive: New Status in Ubuntu Cloud Archive antelope series: New Status in Ubuntu Cloud Archive ussuri series: In Progress Status in Ubuntu Cloud Archive wallaby series: New Status in Ubuntu Cloud Archive xena series: New Status in Ubuntu Cloud Archive yoga series: New Status in Ubuntu Cloud Archive zed series: New Status in neutron: Fix Released Status in neutron package in Ubuntu: New Status in neutron source package in Focal: New Status in neutron source package in Jammy: New Status in neutron source package in Lunar: New Bug description: The port "uplink_status_propagation" feature does not work when OVN is used as the mechanism driver. The reproducer below is working fine with openvswitch as the mechanism driver, but not with the OVN: openstack port create --binding-profile trusted=true --enable-uplink- status-propagation --net private --vnic-type direct test-sriov-bond- enable-uplink-status-propagation-vm-1-port-1 The command fails with the following error when OVN is the mech driver: BadRequestException: 400: Client Error for url: https://10.5.3.81:9696/v2.0/ports, Unrecognized attribute(s) 'propagate_uplink_status' With ML2/OVS, the port creation command above succeeds without any errors. As for the ml2_conf, "uplink_status_propagation" is listed in the extension drivers: [ml2] extension_drivers=port_security,dns_domain_ports,uplink_status_propagation type_drivers = geneve,gre,vlan,flat,local tenant_network_types = geneve,gre,vlan,flat,local mechanism_drivers = ovn,sriovnicswitch /*...*/ I also found the following document which shows the feature gap between ML2/OVS and OVN, but the uplink_status_propagation is not listed: https://docs.openstack.org/neutron/latest/ovn/gaps.html#id9 , maybe this page can be updated as well. To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/2032770/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 2032770] Re: [OVN] port creation with --enable-uplink-status-propagation does not work with OVN mechanism driver
** No longer affects: cloud-archive/bobcat -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/2032770 Title: [OVN] port creation with --enable-uplink-status-propagation does not work with OVN mechanism driver Status in Ubuntu Cloud Archive: New Status in Ubuntu Cloud Archive antelope series: New Status in Ubuntu Cloud Archive ussuri series: New Status in Ubuntu Cloud Archive wallaby series: New Status in Ubuntu Cloud Archive xena series: New Status in Ubuntu Cloud Archive yoga series: New Status in Ubuntu Cloud Archive zed series: New Status in neutron: Fix Released Status in neutron package in Ubuntu: New Status in neutron source package in Jammy: New Status in neutron source package in Lunar: New Bug description: The port "uplink_status_propagation" feature does not work when OVN is used as the mechanism driver. The reproducer below is working fine with openvswitch as the mechanism driver, but not with the OVN: openstack port create --binding-profile trusted=true --enable-uplink- status-propagation --net private --vnic-type direct test-sriov-bond- enable-uplink-status-propagation-vm-1-port-1 The command fails with the following error when OVN is the mech driver: BadRequestException: 400: Client Error for url: https://10.5.3.81:9696/v2.0/ports, Unrecognized attribute(s) 'propagate_uplink_status' With ML2/OVS, the port creation command above succeeds without any errors. As for the ml2_conf, "uplink_status_propagation" is listed in the extension drivers: [ml2] extension_drivers=port_security,dns_domain_ports,uplink_status_propagation type_drivers = geneve,gre,vlan,flat,local tenant_network_types = geneve,gre,vlan,flat,local mechanism_drivers = ovn,sriovnicswitch /*...*/ I also found the following document which shows the feature gap between ML2/OVS and OVN, but the uplink_status_propagation is not listed: https://docs.openstack.org/neutron/latest/ovn/gaps.html#id9 , maybe this page can be updated as well. To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/2032770/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 2032770] Re: [OVN] port creation with --enable-uplink-status-propagation does not work with OVN mechanism driver
** No longer affects: neutron (Ubuntu Focal) ** No longer affects: cloud-archive/victoria -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/2032770 Title: [OVN] port creation with --enable-uplink-status-propagation does not work with OVN mechanism driver Status in Ubuntu Cloud Archive: New Status in Ubuntu Cloud Archive antelope series: New Status in Ubuntu Cloud Archive bobcat series: New Status in Ubuntu Cloud Archive ussuri series: New Status in Ubuntu Cloud Archive wallaby series: New Status in Ubuntu Cloud Archive xena series: New Status in Ubuntu Cloud Archive yoga series: New Status in Ubuntu Cloud Archive zed series: New Status in neutron: Fix Released Status in neutron package in Ubuntu: New Status in neutron source package in Jammy: New Status in neutron source package in Lunar: New Bug description: The port "uplink_status_propagation" feature does not work when OVN is used as the mechanism driver. The reproducer below is working fine with openvswitch as the mechanism driver, but not with the OVN: openstack port create --binding-profile trusted=true --enable-uplink- status-propagation --net private --vnic-type direct test-sriov-bond- enable-uplink-status-propagation-vm-1-port-1 The command fails with the following error when OVN is the mech driver: BadRequestException: 400: Client Error for url: https://10.5.3.81:9696/v2.0/ports, Unrecognized attribute(s) 'propagate_uplink_status' With ML2/OVS, the port creation command above succeeds without any errors. As for the ml2_conf, "uplink_status_propagation" is listed in the extension drivers: [ml2] extension_drivers=port_security,dns_domain_ports,uplink_status_propagation type_drivers = geneve,gre,vlan,flat,local tenant_network_types = geneve,gre,vlan,flat,local mechanism_drivers = ovn,sriovnicswitch /*...*/ I also found the following document which shows the feature gap between ML2/OVS and OVN, but the uplink_status_propagation is not listed: https://docs.openstack.org/neutron/latest/ovn/gaps.html#id9 , maybe this page can be updated as well. To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/2032770/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 2032770] Re: [OVN] port creation with --enable-uplink-status-propagation does not work with OVN mechanism driver
** Also affects: neutron (Ubuntu) Importance: Undecided Status: New ** Changed in: neutron (Ubuntu) Assignee: (unassigned) => Mustafa Kemal Gilor (mustafakemalgilor) ** Also affects: neutron (Ubuntu Jammy) Importance: Undecided Status: New ** Also affects: neutron (Ubuntu Mantic) Importance: Undecided Assignee: Mustafa Kemal Gilor (mustafakemalgilor) Status: New ** Also affects: neutron (Ubuntu Lunar) Importance: Undecided Status: New ** Also affects: neutron (Ubuntu Focal) Importance: Undecided Status: New ** Also affects: cloud-archive Importance: Undecided Status: New ** Also affects: cloud-archive/yoga Importance: Undecided Status: New ** Also affects: cloud-archive/victoria Importance: Undecided Status: New ** Also affects: cloud-archive/zed Importance: Undecided Status: New ** Also affects: cloud-archive/bobcat Importance: Undecided Status: New ** Also affects: cloud-archive/antelope Importance: Undecided Status: New ** Also affects: cloud-archive/ussuri Importance: Undecided Status: New ** Also affects: cloud-archive/xena Importance: Undecided Status: New ** Also affects: cloud-archive/wallaby Importance: Undecided Status: New ** No longer affects: neutron (Ubuntu Mantic) -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/2032770 Title: [OVN] port creation with --enable-uplink-status-propagation does not work with OVN mechanism driver Status in Ubuntu Cloud Archive: New Status in Ubuntu Cloud Archive antelope series: New Status in Ubuntu Cloud Archive bobcat series: New Status in Ubuntu Cloud Archive ussuri series: New Status in Ubuntu Cloud Archive wallaby series: New Status in Ubuntu Cloud Archive xena series: New Status in Ubuntu Cloud Archive yoga series: New Status in Ubuntu Cloud Archive zed series: New Status in neutron: Fix Released Status in neutron package in Ubuntu: New Status in neutron source package in Jammy: New Status in neutron source package in Lunar: New Bug description: The port "uplink_status_propagation" feature does not work when OVN is used as the mechanism driver. The reproducer below is working fine with openvswitch as the mechanism driver, but not with the OVN: openstack port create --binding-profile trusted=true --enable-uplink- status-propagation --net private --vnic-type direct test-sriov-bond- enable-uplink-status-propagation-vm-1-port-1 The command fails with the following error when OVN is the mech driver: BadRequestException: 400: Client Error for url: https://10.5.3.81:9696/v2.0/ports, Unrecognized attribute(s) 'propagate_uplink_status' With ML2/OVS, the port creation command above succeeds without any errors. As for the ml2_conf, "uplink_status_propagation" is listed in the extension drivers: [ml2] extension_drivers=port_security,dns_domain_ports,uplink_status_propagation type_drivers = geneve,gre,vlan,flat,local tenant_network_types = geneve,gre,vlan,flat,local mechanism_drivers = ovn,sriovnicswitch /*...*/ I also found the following document which shows the feature gap between ML2/OVS and OVN, but the uplink_status_propagation is not listed: https://docs.openstack.org/neutron/latest/ovn/gaps.html#id9 , maybe this page can be updated as well. To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/2032770/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1998789] Re: PooledLDAPHandler.result3 does not release pool connection back when an exception is raised
** Also affects: keystone (Ubuntu) Importance: Undecided Status: New ** Also affects: keystone (Ubuntu Jammy) Importance: Undecided Status: New ** Also affects: keystone (Ubuntu Focal) Importance: Undecided Status: New ** Also affects: cloud-archive Importance: Undecided Status: New ** Also affects: cloud-archive/victoria Importance: Undecided Status: New ** Also affects: cloud-archive/zed Importance: Undecided Status: New ** Also affects: cloud-archive/ussuri Importance: Undecided Status: New ** Also affects: cloud-archive/xena Importance: Undecided Status: New ** Also affects: cloud-archive/wallaby Importance: Undecided Status: New ** Also affects: cloud-archive/yoga Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Identity (keystone). https://bugs.launchpad.net/bugs/1998789 Title: PooledLDAPHandler.result3 does not release pool connection back when an exception is raised Status in Ubuntu Cloud Archive: New Status in Ubuntu Cloud Archive ussuri series: New Status in Ubuntu Cloud Archive victoria series: New Status in Ubuntu Cloud Archive wallaby series: New Status in Ubuntu Cloud Archive xena series: New Status in Ubuntu Cloud Archive yoga series: New Status in Ubuntu Cloud Archive zed series: New Status in OpenStack Identity (keystone): Fix Released Status in keystone package in Ubuntu: New Status in keystone source package in Focal: New Status in keystone source package in Jammy: New Bug description: This is a follow-up issue for LP#1896125. This problem has happened when LDAP connection pooling is on (use_pool=True), page_size > 0 and pool_connection_timeout is < 'ldap server response time'. The scenario is as follows: - An user tries to log in to a domain that is attached to LDAP backend. - LDAP server does not respond in `pool_connection_timeout` seconds, causing LDAP connection to raise a ldap.TIMEOUT() exception - From now on, all subsequent LDAP requests will fail with ldappool.MaxConnectionReachedError An in-depth analysis explains why it happens: - LDAP query initiated for user login request with BaseLdap._ldap_get() function call, which grabs a connection with self.get_connection() and invokes conn.search_s() - conn.search_s() invokes conn._paged_search_s() since page_size is > 0 - conn._paged_search_s() calls conn.search_ext() (PooledLDAPHandler.search_ext) method - conn.search_ext() initiates an asynchronous LDAP request and returns an AsynchronousMessage object to the _paged_search_s(), representing the request. - conn._paged_search_s() tries to obtain asynchronous LDAP request results via calling conn.result3() (PooledLDAPHandler.result3) - conn.result3() calls message.connection.result3() - the server cannot respond in pool_connection_timeout seconds, - message.connection.result3() raises a ldap.TIMEOUT(), causes subsequent connection release function, message.clean() to be not called - the connection is kept active forever, subsequent requests cannot use it anymore Reproducer: - Deploy an LDAP server of your choice - Fill it with many data so the search takes more than `pool_connection_timeout` seconds - Define a keystone domain with the LDAP driver with following options: [ldap] use_pool = True page_size = 100 pool_connection_timeout = 3 pool_retry_max = 3 pool_size = 10 - Point the domain to the LDAP server - Try to login to the OpenStack dashboard, or try to do anything that uses the LDAP user - Observe the /var/log/apache2/keystone_error.log, it should contain ldap.TIMEOUT() stack traces followed by `ldappool.MaxConnectionReachedError` stack traces Known workarounds: - Disable LDAP pooling by setting use_pool=Flase - Set page_size to 0 To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1998789/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1998789] [NEW] PooledLDAPHandler.result3 does not release pool connection back when an exception is raised
Public bug reported: This is a follow-up issue for LP#1896125. This problem has happened when LDAP connection pooling is on (use_pool=True), page_size > 0 and pool_connection_timeout is < 'ldap server response time'. The scenario is as follows: - An user tries to log in to a domain that is attached to LDAP backend. - LDAP server does not respond in `pool_connection_timeout` seconds, causing LDAP connection to raise a ldap.TIMEOUT() exception - From now on, all subsequent LDAP requests will fail with ldappool.MaxConnectionReachedError An in-depth analysis explains why it happens: - LDAP query initiated for user login request with BaseLdap._ldap_get() function call, which grabs a connection with self.get_connection() and invokes conn.search_s() - conn.search_s() invokes conn._paged_search_s() since page_size is > 0 - conn._paged_search_s() calls conn.search_ext() (PooledLDAPHandler.search_ext) method - conn.search_ext() initiates an asynchronous LDAP request and returns an AsynchronousMessage object to the _paged_search_s(), representing the request. - conn._paged_search_s() tries to obtain asynchronous LDAP request results via calling conn.result3() (PooledLDAPHandler.result3) - conn.result3() calls message.connection.result3() - the server cannot respond in pool_connection_timeout seconds, - message.connection.result3() raises a ldap.TIMEOUT(), causes subsequent connection release function, message.clean() to be not called - the connection is kept active forever, subsequent requests cannot use it anymore Reproducer: - Deploy an LDAP server of your choice - Fill it with many data so the search takes more than `pool_connection_timeout` seconds - Define a keystone domain with the LDAP driver with following options: [ldap] use_pool = True page_size = 100 pool_connection_timeout = 3 pool_retry_max = 3 pool_size = 10 - Point the domain to the LDAP server - Try to login to the OpenStack dashboard, or try to do anything that uses the LDAP user - Observe the /var/log/apache2/keystone_error.log, it should contain ldap.TIMEOUT() stack traces followed by `ldappool.MaxConnectionReachedError` stack traces Known workarounds: - Disable LDAP pooling by setting use_pool=Flase - Set page_size to 0 ** Affects: keystone Importance: Undecided Assignee: Mustafa Kemal Gilor (mustafakemalgilor) Status: New ** Tags: sts ** Summary changed: - PooledLDAPHandler.result3 does not release pool connection back when an exception raises + PooledLDAPHandler.result3 does not release pool connection back when an exception is raised ** Changed in: keystone Assignee: (unassigned) => Mustafa Kemal Gilor (mustafakemalgilor) -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Identity (keystone). https://bugs.launchpad.net/bugs/1998789 Title: PooledLDAPHandler.result3 does not release pool connection back when an exception is raised Status in OpenStack Identity (keystone): New Bug description: This is a follow-up issue for LP#1896125. This problem has happened when LDAP connection pooling is on (use_pool=True), page_size > 0 and pool_connection_timeout is < 'ldap server response time'. The scenario is as follows: - An user tries to log in to a domain that is attached to LDAP backend. - LDAP server does not respond in `pool_connection_timeout` seconds, causing LDAP connection to raise a ldap.TIMEOUT() exception - From now on, all subsequent LDAP requests will fail with ldappool.MaxConnectionReachedError An in-depth analysis explains why it happens: - LDAP query initiated for user login request with BaseLdap._ldap_get() function call, which grabs a connection with self.get_connection() and invokes conn.search_s() - conn.search_s() invokes conn._paged_search_s() since page_size is > 0 - conn._paged_search_s() calls conn.search_ext() (PooledLDAPHandler.search_ext) method - conn.search_ext() initiates an asynchronous LDAP request and returns an AsynchronousMessage object to the _paged_search_s(), representing the request. - conn._paged_search_s() tries to obtain asynchronous LDAP request results via calling conn.result3() (PooledLDAPHandler.result3) - conn.result3() calls message.connection.result3() - the server cannot respond in pool_connection_timeout seconds, - message.connection.result3() raises a ldap.TIMEOUT(), causes subsequent connection release function, message.clean() to be not called - the connection is kept active forever, subsequent requests cannot use it anymore Reproducer: - Deploy an LDAP server of your choice - Fill it with many data so the search takes more than `pool_connection_timeout` seconds - Define a keystone domain with the LDAP driver with following options: [ldap] use_pool = True page_size = 100 pool_connection_timeout = 3 pool_retry_max = 3 pool_size = 10 - Point the doma
