Re: [Openstack] [Swift]A design draft of Storage Quota

[Everett Toews]

I already posted this over in dev but thought I'd tie off this thread too.

I've just proposed a design summit session with the description below. I 
sincerely hope anyone interested in quotas will attend. Please have a look at 
the etherpad to see what's already been done.

"The quota situation is getting out of control. There are already a number of 
quota implementations for Swift. I expect that Glance will get its own 
implementation of quotas next followed closely by Quantum and probably Cinder. 
Each service will have its own implementation and different API to boot. This 
is definitely suboptimal.

We need centralized quotas, otherwise it will become an operations nightmare.

This session will be focused on how that goal can be achieved and who would be 
willing and able to contribute to it.

I've started an Etherpad at 
https://etherpad.openstack.org/HavanaCentralizedQuotas for this session and to 
track the existing efforts on quotas. Feel free to add any implementations I've 
missed.

To keep the session focused we won't be discussing Nova quotas. If we can nail 
down a way forward for centralized quotas, perhaps one day we can roll Nova 
quotas into it."

Regards,
Everett

On Feb 20, 2013, at 2:43 PM, Everett Toews wrote:

On 2/20/13 1:28 PM, "Tim Bell" mailto:tim.b...@cern.ch>> 
wrote:

We also feel an integrated approach such as Boson is the way forward for
quota rather than each project have its own (and potentially differing)
implementations on areas such as delegation.

+1 to an integrated approach.

It's easy to envision a need for quotas in Glance as well, in both public
and private cloud scenarios.

Cc'ing some other potentially interested parties.

Everett


___
Mailing list: https://launchpad.net/~openstack
Post to : 
openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [Swift]A design draft of Storage Quota

[Thierry Carrez]

Alex Yang wrote:
>  Storage Quotas Design
> 
> This is the design draft of Storage Quota.
> Implementation of this design is
> https://github.com/AlexYangYu/StackLab-swift/tree/dev-quota

Looks like a discussion that should have happened on
openstack-...@lists.openstack.org, since it is about future development
rather than the current state of affairs.

-- 
Thierry Carrez (ttx)
On-topic mailing-list police

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [Swift]A design draft of Storage Quota

[Everett Toews]

On 2/20/13 1:28 PM, "Tim Bell"  wrote:

>We also feel an integrated approach such as Boson is the way forward for
>quota rather than each project have its own (and potentially differing)
>implementations on areas such as delegation.

+1 to an integrated approach.

It's easy to envision a need for quotas in Glance as well, in both public
and private cloud scenarios.

Cc'ing some other potentially interested parties.

Everett


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [Swift]A design draft of Storage Quota

[Chmouel Boudjnah]

I was guessing this is the part of what synaps do but from looking at
https://wiki.openstack.org/wiki/Synaps it may just provide only
notifications and not enforcements.

Chmouel.

On Wed, Feb 20, 2013 at 9:17 PM, Kevin L. Mitchell
 wrote:
> On Wed, 2013-02-20 at 21:09 +0100, Chmouel Boudjnah wrote:
>> On Wed, Feb 20, 2013 at 5:26 PM, Kevin L. Mitchell
>>  wrote:
>> > I'll also point out Boson: https://wiki.openstack.org/wiki/Boson and
>> > https://github.com/klmitch/boson with some initial work.  Unfortunately,
>> > I'm not able to work on Boson at the moment due to higher-priority
>> > tasks…
>>
>> From a quick look of it why can't we do the same as Boson without
>> synaps[1]+ceilometer+swift_container_update. I don't know very well
>> those but from the look of it you could have synaps generating alerts
>> based on resources collection from ceilometer and set the enforcement
>> using the native service mechanism?
>
> I do not understand your question.  Quotas have nothing to do with
> notifications, as far as I understand it; quotas limit the maximum
> amount of a given resource a given user can have, while ceilometer just
> notifies other consumers about actions, right?
> --
> Kevin L. Mitchell 
>
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [Swift]A design draft of Storage Quota

[Kevin L. Mitchell]

On Wed, 2013-02-20 at 21:09 +0100, Chmouel Boudjnah wrote:
> On Wed, Feb 20, 2013 at 5:26 PM, Kevin L. Mitchell
>  wrote:
> > I'll also point out Boson: https://wiki.openstack.org/wiki/Boson and
> > https://github.com/klmitch/boson with some initial work.  Unfortunately,
> > I'm not able to work on Boson at the moment due to higher-priority
> > tasks…
> 
> From a quick look of it why can't we do the same as Boson without
> synaps[1]+ceilometer+swift_container_update. I don't know very well
> those but from the look of it you could have synaps generating alerts
> based on resources collection from ceilometer and set the enforcement
> using the native service mechanism?

I do not understand your question.  Quotas have nothing to do with
notifications, as far as I understand it; quotas limit the maximum
amount of a given resource a given user can have, while ceilometer just
notifies other consumers about actions, right?
-- 
Kevin L. Mitchell 


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [Swift]A design draft of Storage Quota

[Chmouel Boudjnah]

Hi Kevin,

On Wed, Feb 20, 2013 at 5:26 PM, Kevin L. Mitchell
 wrote:
> I'll also point out Boson: https://wiki.openstack.org/wiki/Boson and
> https://github.com/klmitch/boson with some initial work.  Unfortunately,
> I'm not able to work on Boson at the moment due to higher-priority
> tasks…

>From a quick look of it why can't we do the same as Boson without
synaps[1]+ceilometer+swift_container_update. I don't know very well
those but from the look of it you could have synaps generating alerts
based on resources collection from ceilometer and set the enforcement
using the native service mechanism?

Chmouel,

[1] when properly working with a proper API.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [Swift]A design draft of Storage Quota

[Tim Bell]

We also feel an integrated approach such as Boson is the way forward for quota 
rather than each project have its own (and potentially differing) 
implementations on areas such as delegation.

Tim

> -Original Message-
> From: openstack-bounces+tim.bell=cern...@lists.launchpad.net
> [mailto:openstack-bounces+tim.bell=cern...@lists.launchpad.net] On Behalf Of
> Kevin L. Mitchell
> Sent: 20 February 2013 17:26
> To: openstack@lists.launchpad.net
> Subject: Re: [Openstack] [Swift]A design draft of Storage Quota
> 
> On Wed, 2013-02-20 at 18:11 +0800, Alex Yang wrote:
> >  Storage Quotas Design
> > This is the design draft of Storage Quota.
> > Implementation of this design is
> > https://github.com/AlexYangYu/StackLab-swift/tree/dev-quota
> 
> I'll also point out Boson: https://wiki.openstack.org/wiki/Boson and
> https://github.com/klmitch/boson with some initial work.  Unfortunately, I'm
> not able to work on Boson at the moment due to higher-priority tasks…
> --
> Kevin L. Mitchell 
> 
> 
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp


smime.p7s
Description: S/MIME cryptographic signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [Swift]A design draft of Storage Quota

[Kevin L. Mitchell]

On Wed, 2013-02-20 at 18:11 +0800, Alex Yang wrote:
>  Storage Quotas Design
> This is the design draft of Storage Quota.
> Implementation of this design is
> https://github.com/AlexYangYu/StackLab-swift/tree/dev-quota

I'll also point out Boson: https://wiki.openstack.org/wiki/Boson and
https://github.com/klmitch/boson with some initial work.  Unfortunately,
I'm not able to work on Boson at the moment due to higher-priority
tasks…
-- 
Kevin L. Mitchell 


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [Swift]A design draft of Storage Quota

[Alex Yang]

2013/2/20 Chmouel Boudjnah 

> I haven't had the chance to do a full reviews but here is a few toughts :
>
> - the account quota goes along this review:
>
> https://review.openstack.org/#/c/21563/
>
> so we can get easily have account metadatas from middlewares.
>
Yes, I also want quota-level to be stored in memcache through the
account_info() function.

>
> - I am not sure I fancy much the json blob in the ini config (why not a
> new config file?)
>
It's a good suggestion.

>
> - I have already an implementation that mostly follow the same structure
> as (the already merged)  container quota middleware but I am waiting on the
> other review for account info metadata to send it for reviews.
>
> - Down the line but probably not for the v1 I like to be able to notify
> somewhere about the account being over quota ideally it would be an option
> to get to ceilometer or other plugged in system.
>

> Chmouel.
>
>
> On Wed, Feb 20, 2013 at 11:11 AM, Alex Yang  wrote:
>
>>
>>  Storage Quotas 
>> Design
>> This is the design draft of Storage Quota.
>> Implementation of this design is
>> https://github.com/AlexYangYu/StackLab-swift/tree/dev-quota
>>
>>
>> --
>>   杨雨
>>   Email:   alex890...@gmail.com
>> GitHub:   https://github.com/AlexYangYu
>> Blog:http://alexyang.sinaapp.com
>>  Weibo:   http://www.weibo.com/alexyangyu
>>
>> ___
>> Mailing list: https://launchpad.net/~openstack
>> Post to : openstack@lists.launchpad.net
>> Unsubscribe : https://launchpad.net/~openstack
>> More help   : https://help.launchpad.net/ListHelp
>>
>>
>


-- 
  杨雨
  Email:   alex890...@gmail.com
GitHub:   https://github.com/AlexYangYu
Blog:http://alexyang.sinaapp.com
 Weibo:   http://www.weibo.com/alexyangyu
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [Swift]A design draft of Storage Quota

[Alex Yang]

Hi Schwede,
I have already read your codes. To implement the authentication of updating
metadata in the middleware is good for maintainance.
But the account usage is not accurate because of the eventual consistency.
So I do the usage quota based on container.
I will refactor my implementation collaborate with your work and Boudjnah's
work.

Thanks!
.


2013/2/20 Christian Schwede 

> Hi,
> you might also have a look at 
> https://github.com/cschwede/**swquota
>
> Account quota is stored in account metadata and set by a reseller account.
> I changed the code slightly to make it more consistent to the already
> merged container quota.
>
> @chmouel: I might create a pull request to swift itself - what do you
> think?
>
> Christian
>
>
> Am Mittwoch, den 20.02.2013, 11:11 +0100 schrieb Alex Yang <
> alex890...@gmail.com>:
>
>>  Storage Quotas Design [1]
>>
>> This is the design draft of Storage Quota.
>> Implementation of this design is
>> https://github.com/AlexYangYu/**StackLab-swift/tree/dev-quota[2]
>>
>
>


-- 
  杨雨
  Email:   alex890...@gmail.com
GitHub:   https://github.com/AlexYangYu
Blog:http://alexyang.sinaapp.com
 Weibo:   http://www.weibo.com/alexyangyu
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [Swift]A design draft of Storage Quota

[Christian Schwede]

Hi,
you might also have a look at https://github.com/cschwede/swquota

Account quota is stored in account metadata and set by a reseller 
account.
I changed the code slightly to make it more consistent to the already 
merged container quota.


@chmouel: I might create a pull request to swift itself - what do you 
think?


Christian


Am Mittwoch, den 20.02.2013, 11:11 +0100 schrieb Alex Yang 
:

 Storage Quotas Design [1]
This is the design draft of Storage Quota.
Implementation of this design is
https://github.com/AlexYangYu/StackLab-swift/tree/dev-quota [2]



___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [Swift]A design draft of Storage Quota

[Chmouel Boudjnah]

I haven't had the chance to do a full reviews but here is a few toughts :

- the account quota goes along this review:

https://review.openstack.org/#/c/21563/

so we can get easily have account metadatas from middlewares.

- I am not sure I fancy much the json blob in the ini config (why not a new
config file?)

- I have already an implementation that mostly follow the same structure as
(the already merged)  container quota middleware but I am waiting on the
other review for account info metadata to send it for reviews.

- Down the line but probably not for the v1 I like to be able to notify
somewhere about the account being over quota ideally it would be an option
to get to ceilometer or other plugged in system.

Chmouel.


On Wed, Feb 20, 2013 at 11:11 AM, Alex Yang  wrote:

>
>  Storage Quotas 
> Design
> This is the design draft of Storage Quota.
> Implementation of this design is
> https://github.com/AlexYangYu/StackLab-swift/tree/dev-quota
>
>
> --
>   杨雨
>   Email:   alex890...@gmail.com
> GitHub:   https://github.com/AlexYangYu
> Blog:http://alexyang.sinaapp.com
>  Weibo:   http://www.weibo.com/alexyangyu
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp