This bug was fixed in the package keystone - 2:9.3.0-0ubuntu3.2 --------------- keystone (2:9.3.0-0ubuntu3.2) xenial; urgency=medium
* LDAP backend performance improvements (LP: #1582585) - d/p/prevent-error-when-duplicate-mapping-is-created.patch: Handle races for creating id mappings. - d/p/added-cache-for-id-mapping-manager.patch: Add a cache to the id mapping manager to improve performance. - d/p/add-mapping_populate-command.patch: Add a keystone-manage command to populate id mappings between backend identity provider and keystone database. - d/p/faster-id-mapping-lookup.patch: Allow querying for all public ids in a domain at once instead of N queries (one per entity). - d/p/fallback-for-custom-id-map-drivers.patch: Add fallback path for faster-id-mapping lookup for any customer id mapping drivers that may be in use or existing deployments. -- Billy Olsen <billy.ol...@canonical.com> Wed, 10 Jan 2018 14:24:36 -0700 ** Changed in: keystone (Ubuntu Xenial) Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Identity (keystone). https://bugs.launchpad.net/bugs/1582585 Title: the speed of query user from ldap server is very slow Status in Ubuntu Cloud Archive: Invalid Status in Ubuntu Cloud Archive mitaka series: Fix Committed Status in Ubuntu Cloud Archive newton series: Fix Committed Status in OpenStack Identity (keystone): Fix Released Status in keystone package in Ubuntu: Invalid Status in keystone source package in Xenial: Fix Released Bug description: [Impact] * When using an LDAP backend for Keystone, the performance can be slow if there are a large number of users using the cloud. This is due in large part to querying the SQL database for the identity mapping information of each user in a separate transaction. For example, an environment with 12,000 users will result in 12,000 sql queries to the backend database in order to fulfill a user list request. This causes some admin functions in Horizon UI to take several minutes, which often exceeds the WSGI and any haproxy timeouts configured. * This is fixed by backporting a series of patches which caches previously fetched identity mapping information in a memcached instance and changes the logic to query all of the user id mapping by the domain the id mapping is in. Additionally, the keystone-manage command to sync the id mapping information with a backend database in an offline manner is included to allow offline syncing of the data. [Test Case] * Install keystone using an ldap backend w/ large number of users. * List user information: openstack user list --domain <domain_id> * observe slow down [Regression Potential] * For Mitaka, the caching backends such as memcached or mongodb will likely see more usage and an increased footprint due to additional data being cached. Caching the identity mapping information is now standard since Newton and no major issues have been seen coming from this. * This code affects the identity mapping between keystone user and the ldap user (essentially the bridge between the two). While it does not functionally alter the information that is mapped (e.g. no difference in how the identity mapping is calculated), it does alter a key code path for information regarding user identity mappings. [Other Info] * These patches have been run and tested in a staging environment to production and have had exposure in the Mitaka path for approximately one month to show their stability. [Original Description] In our project, the speed of query user from ldap server is very slow,our ldap user number is 12,000,the query costs almost 45 seconds The reason is that keystone will generate the uuid for the ldap users one by one and insert db.And second query time later,it also goes to db,not use the cache. So adding the cache to improve the query speed After adding @MEMOIZE to the following function https://github.com/openstack/keystone/blob/master/keystone/identity/core.py#L580. First query time almost costs 50 seconds,but second query time later it only costs 7 seconds. So it is very necessary to improve this feature To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1582585/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp