[Bug 1582585] Re: the speed of query user from ldap server is very slow
This bug was fixed in the package keystone - 2:9.3.0-0ubuntu3.2~cloud0 --- keystone (2:9.3.0-0ubuntu3.2~cloud0) trusty-mitaka; urgency=medium . * New update for the Ubuntu Cloud Archive. . keystone (2:9.3.0-0ubuntu3.2) xenial; urgency=medium . * LDAP backend performance improvements (LP: #1582585) - d/p/prevent-error-when-duplicate-mapping-is-created.patch: Handle races for creating id mappings. - d/p/added-cache-for-id-mapping-manager.patch: Add a cache to the id mapping manager to improve performance. - d/p/add-mapping_populate-command.patch: Add a keystone-manage command to populate id mappings between backend identity provider and keystone database. - d/p/faster-id-mapping-lookup.patch: Allow querying for all public ids in a domain at once instead of N queries (one per entity). - d/p/fallback-for-custom-id-map-drivers.patch: Add fallback path for faster-id-mapping lookup for any customer id mapping drivers that may be in use or existing deployments. ** Changed in: cloud-archive/mitaka Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1582585 Title: the speed of query user from ldap server is very slow To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1582585/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1582585] Re: the speed of query user from ldap server is very slow
This bug was fixed in the package keystone - 2:10.0.3-0ubuntu1~cloud1 --- keystone (2:10.0.3-0ubuntu1~cloud1) xenial-newton; urgency=medium . * LDAP backend performance improvements (LP: #1582585) - d/p/faster-id-mapping-lookup.patch: Allow querying for all public ids in a domain at once instead of N queries (one per entity). - d/p/fallback-for-custom-id-map-drivers.patch: Add fallback path for faster-id-mapping lookup for any customer id mapping drivers that may be in use or existing deployments. ** Changed in: cloud-archive/newton Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1582585 Title: the speed of query user from ldap server is very slow To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1582585/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1582585] Re: the speed of query user from ldap server is very slow
Regression testing for newton and mitaka were successful: xenial-newton-proposed: Ran: 102 tests in 1448.5591 sec. - Passed: 94 - Skipped: 8 - Expected Fail: 0 - Unexpected Success: 0 - Failed: 0 Sum of execute time for each test: 919.5493 sec. xenial-mitaka-proposed: Ran: 102 tests in 1031.4567 sec. - Passed: 94 - Skipped: 8 - Expected Fail: 0 - Unexpected Success: 0 - Failed: 0 Sum of execute time for each test: 621.0117 sec. trusty-mitaka-proposed: Ran: 102 tests in 1036.5808 sec. - Passed: 94 - Skipped: 8 - Expected Fail: 0 - Unexpected Success: 0 - Failed: 0 Sum of execute time for each test: 603.3084 sec. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1582585 Title: the speed of query user from ldap server is very slow To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1582585/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1582585] Re: the speed of query user from ldap server is very slow
This bug was fixed in the package keystone - 2:9.3.0-0ubuntu3.2 --- keystone (2:9.3.0-0ubuntu3.2) xenial; urgency=medium * LDAP backend performance improvements (LP: #1582585) - d/p/prevent-error-when-duplicate-mapping-is-created.patch: Handle races for creating id mappings. - d/p/added-cache-for-id-mapping-manager.patch: Add a cache to the id mapping manager to improve performance. - d/p/add-mapping_populate-command.patch: Add a keystone-manage command to populate id mappings between backend identity provider and keystone database. - d/p/faster-id-mapping-lookup.patch: Allow querying for all public ids in a domain at once instead of N queries (one per entity). - d/p/fallback-for-custom-id-map-drivers.patch: Add fallback path for faster-id-mapping lookup for any customer id mapping drivers that may be in use or existing deployments. -- Billy OlsenWed, 10 Jan 2018 14:24:36 -0700 ** Changed in: keystone (Ubuntu Xenial) Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1582585 Title: the speed of query user from ldap server is very slow To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1582585/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1582585] Re: the speed of query user from ldap server is very slow
** Tags removed: verification-newton-done verification-xenial-done ** Tags added: verification-done-newton verification-done-xenial -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1582585 Title: the speed of query user from ldap server is very slow To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1582585/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1582585] Re: the speed of query user from ldap server is very slow
Tests completed below with 12,500 users in ldap tree === Mitaka Results === $ dpkg -l | grep keystone ii keystone 2:9.3.0-0ubuntu3.1 all OpenStack identity service - Daemons $ time openstack user list --domain userdomain ... real2m3.608s user0m6.848s sys 0m0.548s Mitaka Proposed $ dpkg -l | grep keystone ii keystone 2:9.3.0-0ubuntu3.2 all OpenStack identity service - Daemons $ time openstack user list --domain userdomain ... real0m16.695s user0m6.708s sys 0m0.328s === Newton Results === $ dpkg -l | grep keystone ii keystone 2:10.0.3-0ubuntu1~cloud0 all OpenStack identity service - Daemons $ time openstack user list --domain userdomain ... real0m22.626s user0m6.232s sys 0m0.492s === Newton-Staging Results === $ dpkg -l | grep keystone ii keystone 2:10.0.3-0ubuntu1~cloud1 all OpenStack identity service - Daemons $ time openstack user list --domain userdomain ... real0m16.297s user0m8.692s sys 0m0.452s -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1582585 Title: the speed of query user from ldap server is very slow To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1582585/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1582585] Re: the speed of query user from ldap server is very slow
Adding tar.gz of /etc/keystone for reference ** Attachment added: "keystone-config.tar.gz" https://bugs.launchpad.net/keystone/+bug/1582585/+attachment/5051668/+files/keystone-config.tar.gz ** Tags removed: verification-mitaka-needed verification-needed verification-needed-xenial verification-newton-needed ** Tags added: verification-done-mitaka verification-newton-done verification-xenial-done -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1582585 Title: the speed of query user from ldap server is very slow To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1582585/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1582585] Re: the speed of query user from ldap server is very slow
@gunph1ld - these changes are already in the ocata code you are running. While 15-20 seconds is still not fast, it is generally faster than the previous code which would issue a query per entry record in the id_mapping table. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1582585 Title: the speed of query user from ldap server is very slow To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1582585/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1582585] Re: the speed of query user from ldap server is very slow
I have the same issue on Ocata (keystone-11.0.3), each request to keystone takes 15-20 sec. There are more then 15 records in the id_mapping table. Does the patch actually fix the problem? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1582585 Title: the speed of query user from ldap server is very slow To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1582585/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1582585] Re: the speed of query user from ldap server is very slow
@raja - can you share your keystone config (with sensitive parts redacted)? How many users are in your ldap directory which is configured as the backend domain? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1582585 Title: the speed of query user from ldap server is very slow To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1582585/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1582585] Re: the speed of query user from ldap server is very slow
and for the mitaka-proposed, looks like the same. the keystone version is: ``` root@ubuntu:~# apt-cache policy keystone keystone: Installed: 2:9.3.0-0ubuntu3.1~cloud0 Candidate: 2:9.3.0-0ubuntu3.1~cloud0 Version table: *** 2:9.3.0-0ubuntu3.1~cloud0 0 500 http://ubuntu-cloud.archive.canonical.com/ubuntu/ trusty-updates/mitaka/main amd64 Packages 100 /var/lib/dpkg/status 1:2014.1.5-0ubuntu1 0 500 http://us.archive.ubuntu.com/ubuntu/ trusty-updates/main amd64 Packages 1:2014.1.3-0ubuntu2.1 0 500 http://security.ubuntu.com/ubuntu/ trusty-security/main amd64 Packages 1:2014.1-0ubuntu1 0 500 http://us.archive.ubuntu.com/ubuntu/ trusty/main amd64 Packages ``` but listing users still doesn't take much time, about 10 seconds: ``` root@ubuntu:~# time openstack user list --domain default real0m9.662s user0m1.284s sys 0m0.232s ``` so seems that I can't reproduce this problem, do you have any suggestions? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1582585 Title: the speed of query user from ldap server is very slow To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1582585/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1582585] Re: the speed of query user from ldap server is very slow
the ubuntu release is: DISTRIB_DESCRIPTION="Ubuntu 16.04.3 LTS" -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1582585 Title: the speed of query user from ldap server is very slow To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1582585/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1582585] Re: the speed of query user from ldap server is very slow
for the newton-proposed: As our ldap server has a limit exceed setting, so I configured ``` [ldap] page_size = 2 ``` before using the keystone in proposed repo, the keystone version is: ``` root@ubuntu:~# apt-cache policy keystone keystone: Installed: 2:10.0.3-0ubuntu1~cloud0 Candidate: 2:10.0.3-0ubuntu1~cloud1 Version table: 2:10.0.3-0ubuntu1~cloud1 500 500 http://ubuntu-cloud.archive.canonical.com/ubuntu xenial-proposed/newton/main amd64 Packages 500 http://ubuntu-cloud.archive.canonical.com/ubuntu xenial-proposed/newton/main i386 Packages *** 2:10.0.3-0ubuntu1~cloud0 500 500 http://ubuntu-cloud.archive.canonical.com/ubuntu xenial-updates/newton/main amd64 Packages 500 http://ubuntu-cloud.archive.canonical.com/ubuntu xenial-updates/newton/main i386 Packages 100 /var/lib/dpkg/status 2:9.3.0-0ubuntu3.1 500 500 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages 500 http://archive.ubuntu.com/ubuntu xenial-updates/main i386 Packages 500 http://archive.ubuntu.com/ubuntu xenial-security/main amd64 Packages 500 http://archive.ubuntu.com/ubuntu xenial-security/main i386 Packages 2:9.0.0-0ubuntu1 500 500 http://archive.ubuntu.com/ubuntu xenial/main amd64 Packages 500 http://archive.ubuntu.com/ubuntu xenial/main i386 Packages ``` list users in ldap will take about 7-11 seconds: ``` root@ubuntu:~# time openstack user list --domain default ... real0m8.522s user0m1.476s sys 0m0.108s ``` after using the package in proposed repo, it takes about 7-11 seconds: ``` root@ubuntu:~# time openstack user list --domain default ... real0m8.637s user0m1.484s sys 0m0.108s ``` I execute above commands for several times, it seems that the patch doesn't have a strong performance improvement -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1582585 Title: the speed of query user from ldap server is very slow To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1582585/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1582585] Re: the speed of query user from ldap server is very slow
Hello jackning, or anyone else affected, Accepted keystone into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/keystone/2:9.3.0-0ubuntu3.2 in a few hours, and then in the -proposed repository. Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users. If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed. Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance! ** Changed in: keystone (Ubuntu Xenial) Status: Triaged => Fix Committed ** Tags added: verification-needed verification-needed-xenial -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1582585 Title: the speed of query user from ldap server is very slow To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1582585/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1582585] Re: the speed of query user from ldap server is very slow
** Description changed: + [Impact] + + * When using an LDAP backend for Keystone, the performance can be slow if there are +a large number of users using the cloud. This is due in large part to querying the +SQL database for the identity mapping information of each user in a separate transaction. +For example, an environment with 12,000 users will result in 12,000 sql queries to the +backend database in order to fulfill a user list request. This causes some admin +functions in Horizon UI to take several minutes, which often exceeds the WSGI and any +haproxy timeouts configured. + + * This is fixed by backporting a series of patches which caches previously fetched identity +mapping information in a memcached instance and changes the logic to query all of the +user id mapping by the domain the id mapping is in. Additionally, the keystone-manage +command to sync the id mapping information with a backend database in an offline manner +is included to allow offline syncing of the data. + + [Test Case] + + * Install keystone using an ldap backend w/ large number of users. + * List user information: openstack user list --domain + * observe slow down + + [Regression Potential] + + * For Mitaka, the caching backends such as memcached or mongodb will likely see more +usage and an increased footprint due to additional data being cached. Caching the +identity mapping information is now standard since Newton and no major issues have +been seen coming from this. + + * This code affects the identity mapping between keystone user and the ldap user +(essentially the bridge between the two). While it does not functionally alter the +information that is mapped (e.g. no difference in how the identity mapping is calculated), +it does alter a key code path for information regarding user identity mappings. + + [Other Info] + + * These patches have been run and tested in a staging environment to production and +have had exposure in the Mitaka path for approximately one month to show their stability. + + [Original Description] + In our project, the speed of query user from ldap server is very slow,our ldap user number is 12,000,the query costs almost 45 seconds The reason is that keystone will generate the uuid for the ldap users one by one and insert db.And second query time later,it also goes to db,not use the cache. So adding the cache to improve the query speed After adding @MEMOIZE to the following function https://github.com/openstack/keystone/blob/master/keystone/identity/core.py#L580. First query time almost costs 50 seconds,but second query time later it only costs 7 seconds. So it is very necessary to improve this feature -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1582585 Title: the speed of query user from ldap server is very slow To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1582585/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1582585] Re: the speed of query user from ldap server is very slow
** Changed in: cloud-archive Status: New => Invalid ** Changed in: keystone (Ubuntu) Status: New => Invalid -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1582585 Title: the speed of query user from ldap server is very slow To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1582585/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1582585] Re: the speed of query user from ldap server is very slow
This is currently building in newton-staging, and will be promoted to newton-proposed once built. I've also uploaded the new package version for xenial to the unapproved queue where it is awaiting review by the SRU team. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1582585 Title: the speed of query user from ldap server is very slow To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1582585/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1582585] Re: the speed of query user from ldap server is very slow
** Tags added: sts -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1582585 Title: the speed of query user from ldap server is very slow To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1582585/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1582585] Re: the speed of query user from ldap server is very slow
The last bullet above is in reference to faster-id-mapping-lookup.patch and fallback-for-custom-id-map-drivers.patch. And patch 1-3 are in reference to: * Patch 1: prevent-error-when-duplicate-mapping-is-created.patch * Patch 2: added-cache-for-id-mapping-manager.patch * Patch 3: add-mapping_populate-command.patch -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1582585 Title: the speed of query user from ldap server is very slow To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1582585/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1582585] Re: the speed of query user from ldap server is very slow
Thanks for the patches Billy. I've reviewed the patches and I think they justify an SRU. Some notes from my review: * Patches 1 through 3 are fairly straight forward cherry-picks that are already in Newton and require a very minimal changes to apply to Mitaka. * Patch 3 adds a new 'mapping_populate' subcommand to keystone-manage that enables creation of id mapping entries. This is a feature, and while we typically don't backport features, I think this case is warranted as it is an optional command that allows an admin to pre-create pubic IDs to prevent users from hitting costly CLI/API calls. * Patch requires additional code to handle backends that don't support get_domain_mapping_list(). I want to make sure testing of the fallback path is performed in addition to the new path. To avoid regression of existing consumers, the OpenStack team will run their continuous integration tests against the packages that are in -proposed. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1582585 Title: the speed of query user from ldap server is very slow To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1582585/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1582585] Re: the speed of query user from ldap server is very slow
The attachment "newton-lp1582585.debdiff" seems to be a debdiff. The ubuntu-sponsors team has been subscribed to the bug report so that they can review and hopefully sponsor the debdiff. If the attachment isn't a patch, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are member of the ~ubuntu-sponsors, unsubscribe the team. [This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issue please contact him.] ** Tags added: patch -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1582585 Title: the speed of query user from ldap server is very slow To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1582585/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1582585] Re: the speed of query user from ldap server is very slow
[Impact] * When using an LDAP backend for Keystone, the performance can be slow if there are a large number of users using the cloud. This is due in large part to querying the SQL database for the identity mapping information of each user in a separate transaction. For example, an environment with 12,000 users will result in 12,000 sql queries to the backend database in order to fulfill a user list request. This causes some admin functions in Horizon UI to take several minutes, which often exceeds the WSGI and any haproxy timeouts configured. * This is fixed by backporting a series of patches which caches previously fetched identity mapping information in a memcached instance and changes the logic to query all of the user id mapping by the domain the id mapping is in. Additionally, the keystone-manage command to sync the id mapping information with a backend database in an offline manner is included to allow offline syncing of the data. [Test Case] * Install keystone using an ldap backend w/ large number of users. * List user information: openstack user list --domain * observe slow down [Regression Potential] * For Mitaka, the caching backends such as memcached or mongodb will likely see more usage and an increased footprint due to additional data being cached. Caching the identity mapping information is now standard since Newton and no major issues have been seen coming from this. * This code affects the identity mapping between keystone user and the ldap user (essentially the bridge between the two). While it does not functionally alter the information that is mapped (e.g. no difference in how the identity mapping is calculated), it does alter a key code path for information regarding user identity mappings. [Other Info] * These patches have been run and tested in a staging environment to production and have had exposure in the Mitaka path for approximately one month to show their stability. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1582585 Title: the speed of query user from ldap server is very slow To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1582585/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1582585] Re: the speed of query user from ldap server is very slow
A patch for Mitaka version (xenial and the trusty-mitaka Ubuntu Cloud Archive) ** Patch added: "mitaka-lp1582585.debdiff" https://bugs.launchpad.net/ubuntu/+source/keystone/+bug/1582585/+attachment/5034743/+files/mitaka-lp1582585.debdiff -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1582585 Title: the speed of query user from ldap server is very slow To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1582585/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1582585] Re: the speed of query user from ldap server is very slow
A patch for xenial-newton Ubuntu Cloud Archive ** Patch added: "newton-lp1582585.debdiff" https://bugs.launchpad.net/ubuntu/+source/keystone/+bug/1582585/+attachment/5034732/+files/newton-lp1582585.debdiff -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1582585 Title: the speed of query user from ldap server is very slow To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1582585/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1582585] Re: the speed of query user from ldap server is very slow
** Also affects: cloud-archive/newton Importance: Undecided Status: New ** Changed in: cloud-archive/newton Status: New => Triaged ** Changed in: cloud-archive/newton Importance: Undecided => High -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1582585 Title: the speed of query user from ldap server is very slow To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1582585/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1582585] Re: the speed of query user from ldap server is very slow
** Also affects: cloud-archive/mitaka Importance: Undecided Status: New ** Also affects: keystone (Ubuntu Xenial) Importance: Undecided Status: New ** Changed in: cloud-archive/mitaka Status: New => Triaged ** Changed in: keystone (Ubuntu Xenial) Status: New => Triaged ** Changed in: cloud-archive/mitaka Importance: Undecided => High ** Changed in: keystone (Ubuntu Xenial) Importance: Undecided => High -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1582585 Title: the speed of query user from ldap server is very slow To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1582585/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1582585] Re: the speed of query user from ldap server is very slow
** Also affects: keystone (Ubuntu) Importance: Undecided Status: New ** Also affects: cloud-archive Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1582585 Title: the speed of query user from ldap server is very slow To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1582585/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs