Re: [openstack-dev] [Horizon] [Performance] Gathering quota usage data in Horizon

[Lingxian Kong]

On Fri, Jan 27, 2017 at 12:28 AM, Rob Cresswell <
robert.cressw...@outlook.com> wrote:

> There's quite a lot to this. So, first off, quotas in horizon are not in
> great shape, and it should be one of our priorities next cycle to improve
> on this. As you've pointed out, it seems any check on quotas right now runs
> multiple serial API calls for everything that has quotas; I haven't checked
> this myself, but others have mentioned the same behaviour.
>
> I don't think anyone is actively working on improving quota behaviour, but
> in the past cycle these two efforts spring to mind:
> - https://blueprints.launchpad.net/horizon/+spec/make-quotas-great-again
> - https://review.openstack.org/#/c/334017/
>
> If there are people with time to work on this effort I'd be happy to
> review. Instances management, quotas, overview pages, Identity work are
> what I'd currently consider the top priorities for improvement.
>

​Thanks Rob for the information. We (Catalyst Cloud) are far more happy to
help with this effort​. I will take a look at the blueprint and check the
latest status of that work with the author.

Cheers,
Lingxian Kong (Larry)
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] [Performance] Gathering quota usage data in Horizon

[Rob Cresswell]

There's quite a lot to this. So, first off, quotas in horizon are not in great 
shape, and it should be one of our priorities next cycle to improve on this. As 
you've pointed out, it seems any check on quotas right now runs multiple serial 
API calls for everything that has quotas; I haven't checked this myself, but 
others have mentioned the same behaviour.

I don't think anyone is actively working on improving quota behaviour, but in 
the past cycle these two efforts spring to mind:
- https://blueprints.launchpad.net/horizon/+spec/make-quotas-great-again
- https://review.openstack.org/#/c/334017/

If there are people with time to work on this effort I'd be happy to review. 
Instances management, quotas, overview pages, Identity work are what I'd 
currently consider the top priorities for improvement.

Rob

On 26 January 2017 at 03:24, Lingxian Kong 
> wrote:
Hi, guys,

Sorry for recalling this thread after 1 year, but we are currently suffering 
from the poor performance issue for our public cloud.

As usage of our customers keeps growing, we are at a stage that should 
seriously pay more attention to horizon performance problem, so Google took me 
to this email after a lot of search.

Currently, when loading a page that may contain some buttons for 
creating/allocating resource (e.g. 'Access & Security'), horizon will check the 
quota usage first to see if a specific button should be disabled or not, and 
the checkings just happen *in sequence*, which makes things even worse.

What's more, the quota usage query in horizon is included in one function[1], 
it will invoke Nova, Cinder, Neutron (perhaps more in future) APIs to get usage 
of bunch of resources, rather than the resource that page is rendering, which 
is another flaw IMHO. I know that this function call is already put in cache, 
but most of our customers' pain just come from the first click.

So, I have a few questions:

1. Does horizon support some config option that could disable quota check? As a 
public cloud, it doesn't make much sense that usage should be limited, and we 
have a monitoring tool that will increase quotas automatically when customer's 
usage will hit the quota limit. So, getting rid of that check will save our 
customers appreciable mass of waiting time.

2. Another option is to support get quota usage for specific resource rather 
than all the resources, e.g. when loading floating ip tab, horizon only get 
floating ip quota usage from Neutron, which has only 2 api calls.

3. I found this FFE[2] which is great (also replied), but splitting tabs is not 
the end, more effort should be put into the performance improvement.

4. Some other trivial improvement like this: https://review.openstack.org/425494

[1]: 
https://github.com/openstack/horizon/blob/master/openstack_dashboard/usage/quotas.py#L396
[2]: 
http://openstack.markmail.org/thread/ra3brm6voo4ouxtx#query:+page:1+mid:oata2tifthnhy5b7+state:results


Cheers,
Lingxian Kong (Larry)

On Wed, Dec 23, 2015 at 9:50 PM, Timur Sufiev 
> wrote:
Duncan,

Thank you for the suggestion, will do.

On Wed, 23 Dec 2015 at 10:55, Duncan Thomas 
> wrote:
On a cloud with a large number of tenants, this is going to involve a large 
number of API calls. I'd suggest you put a spec into cinder to add an API call 
for getting the totals straight out of the DB - it should be easy enough to add.

On 18 December 2015 at 20:35, Timur Sufiev 
> wrote:
Matt,

actually Ivan (Ivan, thanks a lot!) showed me the exact cinderclient call that 
I needed. Now I know how to retrieve Cinder quota usage info per-tenant, seems 
that to retrieve the same info cloud-wide I should sum up all the available 
tenant usages.

With Cinder quota usages being sorted out, my next goal is Nova and Neutron. As 
for Neutron, there are plenty of quota-related calls I'm going to play with 
next week, perhaps there is something suitable for my use case. But as for 
Nova, I haven't found something similar to 'usage' of cinderclient call, so 
help from someone familiar with Nova is very appreciated :).

[0] 
https://github.com/openstack/python-cinderclient/blob/master/cinderclient/v2/quotas.py#L36

On Fri, Dec 18, 2015 at 5:17 PM Matt Riedemann 
> wrote:


On 12/17/2015 2:40 PM, Ivan Kolodyazhny wrote:
> Hi Timur,
>
> Did you try this Cinder API [1]?  Here [2] is cinderclient output.
>
>
>
> [1]
> https://github.com/openstack/python-cinderclient/blob/master/cinderclient/v2/quotas.py#L33
> [2] http://paste.openstack.org/show/482225/
>
> Regards,
> Ivan Kolodyazhny,
> http://blog.e0ne.info/
>
> On Thu, Dec 17, 2015 at 8:41 PM, Timur Sufiev 
> 
> >> wrote:
>
>   

Re: [openstack-dev] [Horizon] [Performance] Gathering quota usage data in Horizon

[Lingxian Kong]

Hi, guys,

Sorry for recalling this thread after 1 year, but we are currently
suffering from the poor performance issue for our public cloud.

As usage of our customers keeps growing, we are at a stage that should
seriously pay more attention to horizon performance problem, so Google took
me to this email after a lot of search.

Currently, when loading a page that may contain some buttons for
creating/allocating resource (e.g. 'Access & Security'), horizon will check
the quota usage first to see if a specific button should be disabled or
not, and the checkings just happen *in sequence*, which makes things even
worse.

What's more, the quota usage query in horizon is included in one
function[1], it will invoke Nova, Cinder, Neutron (perhaps more in future)
APIs to get usage of bunch of resources, rather than the resource that page
is rendering, which is another flaw IMHO. I know that this function call is
already put in cache, but most of our customers' pain just come from the
first click.

So, I have a few questions:

1. Does horizon support some config option that could disable quota check?
As a public cloud, it doesn't make much sense that usage should be limited,
and we have a monitoring tool that will increase quotas automatically when
customer's usage will hit the quota limit. So, getting rid of that check
will save our customers appreciable mass of waiting time.

2. Another option is to support get quota usage for specific resource
rather than all the resources, e.g. when loading floating ip tab, horizon
only get floating ip quota usage from Neutron, which has only 2 api calls.

3. I found this FFE[2] which is great (also replied), but splitting tabs is
not the end, more effort should be put into the performance improvement.

4. Some other trivial improvement like this:
https://review.openstack.org/425494

[1]: https://github.com/openstack/horizon/blob/master/
openstack_dashboard/usage/quotas.py#L396
[2]:
http://openstack.markmail.org/thread/ra3brm6voo4ouxtx#query:+page:1+mid:oata2tifthnhy5b7+state:results


Cheers,
Lingxian Kong (Larry)

On Wed, Dec 23, 2015 at 9:50 PM, Timur Sufiev  wrote:

> Duncan,
>
> Thank you for the suggestion, will do.
>
> On Wed, 23 Dec 2015 at 10:55, Duncan Thomas 
> wrote:
>
>> On a cloud with a large number of tenants, this is going to involve a
>> large number of API calls. I'd suggest you put a spec into cinder to add an
>> API call for getting the totals straight out of the DB - it should be easy
>> enough to add.
>>
>> On 18 December 2015 at 20:35, Timur Sufiev  wrote:
>>
>>> Matt,
>>>
>>> actually Ivan (Ivan, thanks a lot!) showed me the exact cinderclient
>>> call that I needed. Now I know how to retrieve Cinder quota usage info
>>> per-tenant, seems that to retrieve the same info cloud-wide I should sum up
>>> all the available tenant usages.
>>>
>>> With Cinder quota usages being sorted out, my next goal is Nova and
>>> Neutron. As for Neutron, there are plenty of quota-related calls I'm going
>>> to play with next week, perhaps there is something suitable for my use
>>> case. But as for Nova, I haven't found something similar to 'usage' of
>>> cinderclient call, so help from someone familiar with Nova is very
>>> appreciated :).
>>>
>>> [0] https://github.com/openstack/python-cinderclient/blob/ma
>>> ster/cinderclient/v2/quotas.py#L36
>>>
>>> On Fri, Dec 18, 2015 at 5:17 PM Matt Riedemann <
>>> mrie...@linux.vnet.ibm.com> wrote:
>>>


 On 12/17/2015 2:40 PM, Ivan Kolodyazhny wrote:
 > Hi Timur,
 >
 > Did you try this Cinder API [1]?  Here [2] is cinderclient output.
 >
 >
 >
 > [1]
 > https://github.com/openstack/python-cinderclient/blob/master
 /cinderclient/v2/quotas.py#L33
 > [2] http://paste.openstack.org/show/482225/
 >
 > Regards,
 > Ivan Kolodyazhny,
 > http://blog.e0ne.info/
 >
 > On Thu, Dec 17, 2015 at 8:41 PM, Timur Sufiev  > wrote:
 >
 > Hello, folks!
 >
 > I'd like to initiate a discussion of the feature request I'm going
 > to make on behalf of Horizon to every core OpenStack service which
 > supports Quota feature, namely Cinder, Nova and Neutron.
 >
 > Although all three services' APIs support special calls to get
 > current quota limitations (Nova and Cinder allows to get and
 update
 > both per-tenant and default cloud-wide limitations, Neutron allows
 > to do it only for per-tenant limitations), there is no special
 call
 > in any of these services to get current per-tenant usage of quota.
 > Because of that Horizon needs to get, say for 'volumes' quota, a
 > list of Cinder volumes in the current tenant and then just
 calculate
 > its length [1]. When there are really a lot of entities in tenant
 -
 >