Public bug reported:

Short Summary:
In order to rectify negative user experience caused by timeouts associated with 
loading a large amount of objects from the database and displaying them in 
horizon, emulation of pagination within horizon is proposed.


Summary:
When running OpenStack at scale with circa 6000 different projects, horizon 
experiences a timeout when it tries to load project resources (accessing 
through the API works fine). It can be assumed that the issue will hold for 
users and domain alike since the mechanism is the same. The issue is 
reproducible with a large number of users, domains or projects. We believe that 
pagination support in keystone API would solve the aforementioned issues.

Unfortunately, pagination was removed from keystone in the v3 api and as
far as we're aware it was never re-added.

https://docs.openstack.org/api-ref/identity/v3/?expanded=get-available-
project-scopes-detail,list-projects-detail,list-domains-detail#list-
projects

https://docs.openstack.org/api-ref/identity/v3/?expanded=get-available-
project-scopes-detail,list-projects-detail,list-domains-detail#list-
domains

The decision to remove pagination was a conscious and was done around
2015.

https://lists.openstack.org/pipermail/openstack-
dev/2015-August/thread.html#72082

Furthermore, a look at the code confirms the aforementioned deletion of
pagination support. This seems like a fundamental performance oriented
feature because the overhead in data conversion can be quite a bit when
you have a large number of any objects being returned from a database.

https://github.com/openstack/keystone/blame/d7b1d57cae738183f8d85413e942402a8a4efb31/keystone/server/flask/common.py#L675

Currently and effort is under way to get a discussion going in order to
reimplement pagination support within keystone API for domains and
projects, however it is currently not possible to implement the same
feature for users.

Long story short, aforementioned changes to the keystone API lead to a
negative user experience when using Horizon. In order to rectify the
said negative experience in the short run, it is believe possible to
emulate pagination and load users, projects and domains in chunks.

In the long run changes to the keystone API should follow with further
subsequent changes to Horizon for projects and domains, and possibly in
the very long run, users.

** Affects: horizon
     Importance: Undecided
         Status: New

** Summary changed:

- Horizon times out when loading a large amount of projects (>6000)
+ Horizon times out when loading a large amount of resources to display

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Dashboard (Horizon).
https://bugs.launchpad.net/bugs/1982966

Title:
  Horizon times out when loading a large amount of resources to display

Status in OpenStack Dashboard (Horizon):
  New

Bug description:
  Short Summary:
  In order to rectify negative user experience caused by timeouts associated 
with loading a large amount of objects from the database and displaying them in 
horizon, emulation of pagination within horizon is proposed.


  
  Summary:
  When running OpenStack at scale with circa 6000 different projects, horizon 
experiences a timeout when it tries to load project resources (accessing 
through the API works fine). It can be assumed that the issue will hold for 
users and domain alike since the mechanism is the same. The issue is 
reproducible with a large number of users, domains or projects. We believe that 
pagination support in keystone API would solve the aforementioned issues.

  Unfortunately, pagination was removed from keystone in the v3 api and
  as far as we're aware it was never re-added.

  https://docs.openstack.org/api-ref/identity/v3/?expanded=get-
  available-project-scopes-detail,list-projects-detail,list-domains-
  detail#list-projects

  https://docs.openstack.org/api-ref/identity/v3/?expanded=get-
  available-project-scopes-detail,list-projects-detail,list-domains-
  detail#list-domains

  The decision to remove pagination was a conscious and was done around
  2015.

  https://lists.openstack.org/pipermail/openstack-
  dev/2015-August/thread.html#72082

  Furthermore, a look at the code confirms the aforementioned deletion
  of pagination support. This seems like a fundamental performance
  oriented feature because the overhead in data conversion can be quite
  a bit when you have a large number of any objects being returned from
  a database.

  
https://github.com/openstack/keystone/blame/d7b1d57cae738183f8d85413e942402a8a4efb31/keystone/server/flask/common.py#L675

  Currently and effort is under way to get a discussion going in order
  to reimplement pagination support within keystone API for domains and
  projects, however it is currently not possible to implement the same
  feature for users.

  Long story short, aforementioned changes to the keystone API lead to a
  negative user experience when using Horizon. In order to rectify the
  said negative experience in the short run, it is believe possible to
  emulate pagination and load users, projects and domains in chunks.

  In the long run changes to the keystone API should follow with further
  subsequent changes to Horizon for projects and domains, and possibly
  in the very long run, users.

To manage notifications about this bug go to:
https://bugs.launchpad.net/horizon/+bug/1982966/+subscriptions


-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to     : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

Reply via email to