Hi, Would the new library support quota on domain level also? As it stands in oslo-incubator it only does quota enforcement on project level only. The use case for this is quota enforcement across multiple projects. For e.g. as a cloud provider, I would like my customer to create only #X volumes across all his projects.
-Ajaya Cheers, Ajaya On Wed, Oct 15, 2014 at 7:04 PM, Doug Hellmann <d...@doughellmann.com> wrote: > Sigh. Because I typed the wrong command. Thanks for pointing that out. > > I don’t see any instances of “quota” in openstack-common.conf files: > > $ grep quota */openstack-common.conf > > or in any projects under "openstack/“: > > $ ls */*/openstack/common/quota.py > ls: cannot access */*/openstack/common/quota.py: No such file or directory > > I don’t know where manila’s copy came from, but if it has been copied from > the incubator by hand and then changed we should fix that up. > > Doug > > On Oct 15, 2014, at 5:28 AM, Valeriy Ponomaryov <vponomar...@mirantis.com> > wrote: > > But why "policy" is being discussed on "quota" thread? > > On Wed, Oct 15, 2014 at 11:55 AM, Valeriy Ponomaryov < > vponomar...@mirantis.com> wrote: > >> Manila project does use "policy" common code from incubator. >> >> Our "small" wrapper for it: >> https://github.com/openstack/manila/blob/8203c51081680a7a9dba30ae02d7c43d6e18a124/manila/policy.py >> >> >> On Wed, Oct 15, 2014 at 2:41 AM, Salvatore Orlando <sorla...@nicira.com> >> wrote: >> >>> Doug, >>> >>> I totally agree with your findings on the policy module. >>> Neutron already has some "customizations" there and we already have a >>> few contributors working on syncing it back with oslo-incubator during the >>> Kilo release cycle. >>> >>> However, my query was about the quota module. >>> From what I gather it seems not a lot of projects use it: >>> >>> $ find . -name openstack-common.conf | xargs grep quota >>> $ >>> >>> Salvatore >>> >>> On 15 October 2014 00:34, Doug Hellmann <d...@doughellmann.com> wrote: >>> >>>> >>>> On Oct 14, 2014, at 12:31 PM, Salvatore Orlando <sorla...@nicira.com> >>>> wrote: >>>> >>>> Hi Doug, >>>> >>>> do you know if the existing quota oslo-incubator module has already >>>> some active consumers? >>>> In the meanwhile I've pushed a spec to neutron-specs for improving >>>> quota management there [1] >>>> >>>> >>>> It looks like a lot of projects are syncing the module: >>>> >>>> $ grep policy */openstack-common.conf >>>> >>>> barbican/openstack-common.conf:modules=gettextutils,jsonutils,log,local,timeutils,importutils,policy >>>> ceilometer/openstack-common.conf:module=policy >>>> cinder/openstack-common.conf:module=policy >>>> designate/openstack-common.conf:module=policy >>>> gantt/openstack-common.conf:module=policy >>>> glance/openstack-common.conf:module=policy >>>> heat/openstack-common.conf:module=policy >>>> horizon/openstack-common.conf:module=policy >>>> ironic/openstack-common.conf:module=policy >>>> keystone/openstack-common.conf:module=policy >>>> manila/openstack-common.conf:module=policy >>>> neutron/openstack-common.conf:module=policy >>>> nova/openstack-common.conf:module=policy >>>> trove/openstack-common.conf:module=policy >>>> tuskar/openstack-common.conf:module=policy >>>> >>>> I’m not sure how many are actively using it, but I wouldn’t expect them >>>> to copy it in if they weren’t using it at all. >>>> >>>> >>>> Now, I can either work on the oslo-incubator module and leverage it in >>>> Neutron, or develop the quota module in Neutron, and move it to >>>> oslo-incubator once we validate it with Neutron. The latter approach seems >>>> easier from a workflow perspective - as it avoid the intermediate steps of >>>> moving code from oslo-incubator to neutron. On the other hand it will delay >>>> adoption in oslo-incubator. >>>> >>>> >>>> The policy module is up for graduation this cycle. It may end up in its >>>> own library, to allow us to build a review team for the code more easily >>>> than if we put it in with some of the other semi-related modules like the >>>> server code. We’re still working that out [1], and if you expect to make a >>>> lot of incompatible changes we should delay graduation to make that >>>> simpler. >>>> >>>> Either way, since we have so many consumers, I think it would be easier >>>> to have the work happen in Oslo somewhere so we can ensure those changes >>>> are useful to and usable by all of the existing consumers. >>>> >>>> Doug >>>> >>>> [1] https://etherpad.openstack.org/p/kilo-oslo-library-proposals >>>> >>>> >>>> What's your opinion? >>>> >>>> Regards, >>>> Salvatore >>>> >>>> [1] https://review.openstack.org/#/c/128318/ >>>> >>>> On 8 October 2014 18:52, Doug Hellmann <d...@doughellmann.com> wrote: >>>> >>>>> >>>>> On Oct 8, 2014, at 7:03 AM, Davanum Srinivas <dava...@gmail.com> >>>>> wrote: >>>>> >>>>> > Salvatore, Joe, >>>>> > >>>>> > We do have this at the moment: >>>>> > >>>>> > >>>>> https://github.com/openstack/oslo-incubator/blob/master/openstack/common/quota.py >>>>> > >>>>> > — dims >>>>> >>>>> If someone wants to drive creating a useful library during kilo, >>>>> please consider adding the topic to the etherpad we’re using to plan >>>>> summit >>>>> sessions and then come participate in the Oslo meeting this Friday 16:00 >>>>> UTC. >>>>> >>>>> https://etherpad.openstack.org/p/kilo-oslo-summit-topics >>>>> >>>>> Doug >>>>> >>>>> > >>>>> > On Wed, Oct 8, 2014 at 2:29 AM, Salvatore Orlando < >>>>> sorla...@nicira.com> wrote: >>>>> >> >>>>> >> On 8 October 2014 04:13, Joe Gordon <joe.gord...@gmail.com> wrote: >>>>> >>> >>>>> >>> >>>>> >>> On Fri, Oct 3, 2014 at 10:47 AM, Morgan Fainberg >>>>> >>> <morgan.fainb...@gmail.com> wrote: >>>>> >>>> >>>>> >>>> Keeping the enforcement local (same way policy works today) helps >>>>> limit >>>>> >>>> the fragility, big +1 there. >>>>> >>>> >>>>> >>>> I also agree with Vish, we need a uniform way to talk about quota >>>>> >>>> enforcement similar to how we have a uniform policy language / >>>>> enforcement >>>>> >>>> model (yes I know it's not perfect, but it's far closer to >>>>> uniform than >>>>> >>>> quota management is). >>>>> >>> >>>>> >>> >>>>> >>> It sounds like maybe we should have an oslo library for quotas? >>>>> Somewhere >>>>> >>> where we can share the code,but keep the operations local to each >>>>> service. >>>>> >> >>>>> >> >>>>> >> This is what I had in mind as well. A simple library for quota >>>>> enforcement >>>>> >> which can be used regardless of where and how you do it, which >>>>> might depend >>>>> >> on the application business logic, the WSGI framework in use, or >>>>> other >>>>> >> factors. >>>>> >> >>>>> >>> >>>>> >>>> >>>>> >>>> >>>>> >>>> If there is still interest of placing quota in keystone, let's >>>>> talk about >>>>> >>>> how that will work and what will be needed from Keystone . The >>>>> previous >>>>> >>>> attempt didn't get much traction and stalled out early in >>>>> implementation. If >>>>> >>>> we want to revisit this lets make sure we have the resources >>>>> needed and >>>>> >>>> spec(s) in progress / info on etherpads (similar to how the >>>>> multitenancy >>>>> >>>> stuff was handled at the last summit) as early as possible. >>>>> >>> >>>>> >>> >>>>> >>> Why not centralize quota management via the >>>>> python-openstackclient, what >>>>> >>> is the benefit of getting keystone involved? >>>>> >> >>>>> >> >>>>> >> Providing this through the openstack client in my opinion has the >>>>> >> disadvantage that users which either use the REST API direct or >>>>> write their >>>>> >> own clients won't leverage it. I don't think it's a reasonable >>>>> assumption >>>>> >> that everybody will use python-openstackclient, is it? >>>>> >> >>>>> >> Said that, storing quotas in keystone poses a further challenge to >>>>> the >>>>> >> scalability of the system, which we shall perhaps address by using >>>>> >> appropriate caching strategies and leveraging keystone >>>>> notifications. Until >>>>> >> we get that, I think that the openstack client will be the best way >>>>> of >>>>> >> getting a unified quota management experience. >>>>> >> >>>>> >> Salvatore >>>>> >> >>>>> >> >>>>> >>>> >>>>> >>>> Cheers, >>>>> >>>> Morgan >>>>> >>>> >>>>> >>>> Sent via mobile >>>>> >>>> >>>>> >>>> >>>>> >>>> On Friday, October 3, 2014, Salvatore Orlando < >>>>> sorla...@nicira.com> >>>>> >>>> wrote: >>>>> >>>>> >>>>> >>>>> Thanks Vish, >>>>> >>>>> >>>>> >>>>> this seems a very reasonable first step as well - and since most >>>>> >>>>> projects would be enforcing quotas in the same way, the shared >>>>> library would >>>>> >>>>> be the logical next step. >>>>> >>>>> After all this is quite the same thing we do with authZ. >>>>> >>>>> >>>>> >>>>> Duncan is expressing valid concerns which in my opinion can be >>>>> addressed >>>>> >>>>> with an appropriate design - and a decent implementation. >>>>> >>>>> >>>>> >>>>> Salvatore >>>>> >>>>> >>>>> >>>>> On 3 October 2014 18:25, Vishvananda Ishaya < >>>>> vishvana...@gmail.com> >>>>> >>>>> wrote: >>>>> >>>>>> >>>>> >>>>>> The proposal in the past was to keep quota enforcement local, >>>>> but to >>>>> >>>>>> put the resource limits into keystone. This seems like an >>>>> obvious first >>>>> >>>>>> step to me. Then a shared library for enforcing quotas with >>>>> decent >>>>> >>>>>> performance should be next. The quota calls in nova are >>>>> extremely >>>>> >>>>>> inefficient right now and it will only get worse when we try to >>>>> add >>>>> >>>>>> hierarchical projects and quotas. >>>>> >>>>>> >>>>> >>>>>> Vish >>>>> >>>>>> >>>>> >>>>>> On Oct 3, 2014, at 7:53 AM, Duncan Thomas < >>>>> duncan.tho...@gmail.com> >>>>> >>>>>> wrote: >>>>> >>>>>> >>>>> >>>>>>> Taking quota out of the service / adding remote calls for quota >>>>> >>>>>>> management is going to make things fragile - you've somehow >>>>> got to >>>>> >>>>>>> deal with the cases where your quota manager is slow, goes >>>>> away, >>>>> >>>>>>> hiccups, drops connections etc. You'll also need some way of >>>>> >>>>>>> reconciling actual usage against quota usage periodically, to >>>>> detect >>>>> >>>>>>> problems. >>>>> >>>>>>> >>>>> >>>>>>> On 3 October 2014 15:03, Salvatore Orlando < >>>>> sorla...@nicira.com> >>>>> >>>>>>> wrote: >>>>> >>>>>>>> Hi, >>>>> >>>>>>>> >>>>> >>>>>>>> Quota management is currently one of those things where every >>>>> >>>>>>>> openstack >>>>> >>>>>>>> project does its own thing. While quotas are obviously >>>>> managed in a >>>>> >>>>>>>> similar >>>>> >>>>>>>> way for each project, there are subtle differences which >>>>> ultimately >>>>> >>>>>>>> result >>>>> >>>>>>>> in lack of usability. >>>>> >>>>>>>> >>>>> >>>>>>>> I recall that in the past there have been several calls for >>>>> unifying >>>>> >>>>>>>> quota >>>>> >>>>>>>> management. The blueprint [1] for instance, hints at the >>>>> possibility >>>>> >>>>>>>> of >>>>> >>>>>>>> storing quotas in keystone. >>>>> >>>>>>>> On the other hand, the blazar project [2, 3] seems to aim at >>>>> solving >>>>> >>>>>>>> this >>>>> >>>>>>>> problem for good enabling resource reservation and therefore >>>>> >>>>>>>> potentially >>>>> >>>>>>>> freeing openstack projects from managing and enforcing quotas. >>>>> >>>>>>>> >>>>> >>>>>>>> While Blazar is definetely a good thing to have, I'm not >>>>> entirely >>>>> >>>>>>>> sure we >>>>> >>>>>>>> want to make it a "required" component for every deployment. >>>>> Perhaps >>>>> >>>>>>>> single >>>>> >>>>>>>> projects should still be able to enforce quota. On the other >>>>> hand, >>>>> >>>>>>>> at least >>>>> >>>>>>>> on paper, the idea of making Keystone "THE" endpoint for >>>>> managing >>>>> >>>>>>>> quotas, >>>>> >>>>>>>> and then letting the various project enforce them, sounds >>>>> promising >>>>> >>>>>>>> - is >>>>> >>>>>>>> there any reason for which this blueprint is stalled to the >>>>> point >>>>> >>>>>>>> that it >>>>> >>>>>>>> seems forgotten now? >>>>> >>>>>>>> >>>>> >>>>>>>> I'm coming to the mailing list with these random questions >>>>> about >>>>> >>>>>>>> quota >>>>> >>>>>>>> management, for two reasons: >>>>> >>>>>>>> 1) despite developing and using openstack on a daily basis >>>>> I'm still >>>>> >>>>>>>> confused by quotas >>>>> >>>>>>>> 2) I've found a race condition in neutron quotas and the fix >>>>> is not >>>>> >>>>>>>> trivial. >>>>> >>>>>>>> So, rather than start coding right away, it might probably >>>>> make more >>>>> >>>>>>>> sense >>>>> >>>>>>>> to ask the community if there is already a known better >>>>> approach to >>>>> >>>>>>>> quota >>>>> >>>>>>>> management - and obviously enforcement. >>>>> >>>>>>>> >>>>> >>>>>>>> Thanks in advance, >>>>> >>>>>>>> Salvatore >>>>> >>>>>>>> >>>>> >>>>>>>> [1] >>>>> https://blueprints.launchpad.net/keystone/+spec/service-metadata >>>>> >>>>>>>> [2] https://wiki.openstack.org/wiki/Blazar >>>>> >>>>>>>> [3] >>>>> https://review.openstack.org/#/q/project:stackforge/blazar,n,z >>>>> >>>>>>>> >>>>> >>>>>>>> _______________________________________________ >>>>> >>>>>>>> OpenStack-dev mailing list >>>>> >>>>>>>> OpenStack-dev@lists.openstack.org >>>>> >>>>>>>> >>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>>> >>>>>>>> >>>>> >>>>>>> >>>>> >>>>>>> >>>>> >>>>>>> >>>>> >>>>>>> -- >>>>> >>>>>>> Duncan Thomas >>>>> >>>>>>> >>>>> >>>>>>> _______________________________________________ >>>>> >>>>>>> OpenStack-dev mailing list >>>>> >>>>>>> OpenStack-dev@lists.openstack.org >>>>> >>>>>>> >>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>>> >>>>>> >>>>> >>>>>> >>>>> >>>>>> _______________________________________________ >>>>> >>>>>> OpenStack-dev mailing list >>>>> >>>>>> OpenStack-dev@lists.openstack.org >>>>> >>>>>> >>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>>> >>>>>> >>>>> >>>>> >>>>> >>>> >>>>> >>>> _______________________________________________ >>>>> >>>> OpenStack-dev mailing list >>>>> >>>> OpenStack-dev@lists.openstack.org >>>>> >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>>> >>>> >>>>> >>> >>>>> >>> >>>>> >>> _______________________________________________ >>>>> >>> OpenStack-dev mailing list >>>>> >>> OpenStack-dev@lists.openstack.org >>>>> >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>>> >>> >>>>> >> >>>>> >> >>>>> >> _______________________________________________ >>>>> >> OpenStack-dev mailing list >>>>> >> OpenStack-dev@lists.openstack.org >>>>> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>>> >> >>>>> > >>>>> > >>>>> > >>>>> > -- >>>>> > Davanum Srinivas :: https://twitter.com/dims >>>>> > >>>>> > _______________________________________________ >>>>> > OpenStack-dev mailing list >>>>> > OpenStack-dev@lists.openstack.org >>>>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>>> >>>>> >>>>> _______________________________________________ >>>>> OpenStack-dev mailing list >>>>> OpenStack-dev@lists.openstack.org >>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>>> >>>> >>>> _______________________________________________ >>>> OpenStack-dev mailing list >>>> OpenStack-dev@lists.openstack.org >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>> >>>> >>>> >>>> _______________________________________________ >>>> OpenStack-dev mailing list >>>> OpenStack-dev@lists.openstack.org >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>> >>>> >>> >>> _______________________________________________ >>> OpenStack-dev mailing list >>> OpenStack-dev@lists.openstack.org >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>> >>> >> >> >> -- >> Kind Regards >> Valeriy Ponomaryov >> www.mirantis.com >> vponomar...@mirantis.com >> > > > > -- > Kind Regards > Valeriy Ponomaryov > www.mirantis.com > vponomar...@mirantis.com > _______________________________________________ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > _______________________________________________ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >
_______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev