Re: [Openstack-operators] Shelving

[Tim Bell]

On 18 Aug 2016, at 22:48, Michael Still 
mailto:mi...@stillhq.com>> wrote:

Shelved instances still consume IPs and hypervisor disk IIRC, so they're not 
free for nova. So, there has to be some form of accounting of shelved instances 
to stop resource exhaustion, especially in the IP space.

I'm not opposed to removing most of their cost from quota, but I don't think we 
can do it entirely.

Michael


Once offloaded, the only resources consumed are in Glance and the IP 
reservation. I do not know if the IP reservation counts in the Neutron ports 
quota though.

I was aiming to distinguish between quota and cost, I would expect a public 
cloud provider to bill for the space used in Glance to avoid repeated 
snapshotting.

The exact timing of the quota reduction is also for debate. The resource 
provider only sees the benefit when the resource is offloaded but from the user 
perspective, the expectation would be that the quota is available once the user 
request is completed (i.e. shelved). However, the resources are still being 
used at this point until the offload time is reached.

Tim



On Fri, Aug 19, 2016 at 3:50 AM, Tim Bell 
mailto:tim.b...@cern.ch>> wrote:
I was interested to establish a consensus that

- Shelved instances should not be part of the users quota
- Quota in Glance (and associated chargeback if appropriate) is needed

Glance space for us is much less expensive than people leaving their instances 
running. Equally, terminating a user’s inactive VM would not be popular so 
giving them a shelved instance would allow them to re-create it much more 
easily.

Any objections to a blueprint that proposes shelving should be handled with the 
same quota model as snapshotting ?

Tim

On 18/08/16 19:43, "Jonathan D. Proulx" 
mailto:j...@csail.mit.edu>> wrote:

On Thu, Aug 18, 2016 at 03:24:28PM +, Tim Bell wrote:
:
:We’re having a look at VM shelving for the CERN community and struggling 
to find a motivation for a private cloud user to shelve their instances (and 
free up resources they may be only using infrequently).
:
:The problem is that shelved instances seem to still be included in the 
user’s quota. Without internal billing, the best motivation for users to shelve 
would be to allow them to maximize the use of their quota.
:
:Have any other sites used shelving extensively ? How did you motivate your 
users to shelve unused resources?


Hi Tim,

We've just started looking at this and for simialar reasons.  I agree
we should remove shelved resources from project quota. Shelved
instances do still hold some storage resources so there may need to be
new quota to accoutn for that some how...

Currently the only motivation for our users to shelve is to get me to
stop pestering them.

We're considering policy based enforced shelving (based on some yet to
be defined utilization metrics) but that's only a idea at this point
not a plan.

-Jon


___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators



--
Rackspace Australia

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Shelving

[David Medberry]

On Thu, Aug 18, 2016 at 6:43 PM, David Medberry 
wrote:

> Shelve completely dissociates from a node.


and when you unshelve it goes to a node (likely a new node but potentially
the same one). The scheduling algorithm runs as if the instance had never
been on the node. There does seem to be some retention of NUMA state (as
we've failed to use it as a way to bypass the initial NUMA affiliation.)
But it will go on other nodes (of the same NUMA style.) This probably is
irrelevant if you don't use NUMA in the first place (more of a bug in our
case.)
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Shelving

[David Medberry]

Shelve completely dissociates from a node.

On Thu, Aug 18, 2016 at 4:21 PM, Kris G. Lindgren 
wrote:

> Does shelving an instance also free up the instances reservation against
> that node?  If it doesn’t I assume that’s why it still counts against their
> quota?  IE Nova is still trying to keep a slot open for them on that
> server, so when you unshelv does it go back to the same node or does it go
> to a new node?
>
>
>
> ___
>
> Kris Lindgren
>
> Senior Linux Systems Engineer
>
> GoDaddy
>
>
>
> *From: *David Medberry 
> *Date: *Thursday, August 18, 2016 at 3:49 PM
> *To: *"Jonathan D. Proulx" 
> *Cc: *"openstack-operators@lists.openstack.org" <
> openstack-operators@lists.openstack.org>
> *Subject: *Re: [Openstack-operators] Shelving
>
>
>
>
>
> On Thu, Aug 18, 2016 at 3:12 PM, Jonathan D. Proulx 
> wrote:
>
>
> True they do consume IPs.
>
> In my configuration they do not consume any hypervisor disk.  I
> *think* this is true of all configurations once the 'shelved' systems
> are 'offloaded'.
>
>
> i concur, that's the intent of shelving as I understand it, to free up the
> hypervisor by moving off of the hypervisor entirely. So in our case with
> libvirt, there are no "shut off" instance names listed with "virsh list
> --all". They truly are "shelved" with just glance storage (once they are
> "shelve-offload"ed).
>
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Shelving

[Kris G. Lindgren]

Does shelving an instance also free up the instances reservation against that 
node?  If it doesn’t I assume that’s why it still counts against their quota?  
IE Nova is still trying to keep a slot open for them on that server, so when 
you unshelv does it go back to the same node or does it go to a new node?

___
Kris Lindgren
Senior Linux Systems Engineer
GoDaddy

From: David Medberry 
Date: Thursday, August 18, 2016 at 3:49 PM
To: "Jonathan D. Proulx" 
Cc: "openstack-operators@lists.openstack.org" 

Subject: Re: [Openstack-operators] Shelving


On Thu, Aug 18, 2016 at 3:12 PM, Jonathan D. Proulx 
mailto:j...@csail.mit.edu>> wrote:

True they do consume IPs.

In my configuration they do not consume any hypervisor disk.  I
*think* this is true of all configurations once the 'shelved' systems
are 'offloaded'.

i concur, that's the intent of shelving as I understand it, to free up the 
hypervisor by moving off of the hypervisor entirely. So in our case with 
libvirt, there are no "shut off" instance names listed with "virsh list --all". 
They truly are "shelved" with just glance storage (once they are 
"shelve-offload"ed).
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Shelving

[David Medberry]

On Thu, Aug 18, 2016 at 3:12 PM, Jonathan D. Proulx 
wrote:

>
> True they do consume IPs.
>
> In my configuration they do not consume any hypervisor disk.  I
> *think* this is true of all configurations once the 'shelved' systems
> are 'offloaded'.


i concur, that's the intent of shelving as I understand it, to free up the
hypervisor by moving off of the hypervisor entirely. So in our case with
libvirt, there are no "shut off" instance names listed with "virsh list
--all". They truly are "shelved" with just glance storage (once they are
"shelve-offload"ed).
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Shelving

[Jonathan D. Proulx]

On Fri, Aug 19, 2016 at 06:48:12AM +1000, Michael Still wrote:
:Shelved instances still consume IPs and hypervisor disk IIRC, so they're
:not free for nova. So, there has to be some form of accounting of shelved
:instances to stop resource exhaustion, especially in the IP space.
:
:I'm not opposed to removing most of their cost from quota, but I don't
:think we can do it entirely.

True they do consume IPs.

In my configuration they do not consume any hypervisor disk.  I
*think* this is true of all configurations once the 'shelved' systems
are 'offloaded'.

-Jon


___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Shelving

[Michael Still]

Shelved instances still consume IPs and hypervisor disk IIRC, so they're
not free for nova. So, there has to be some form of accounting of shelved
instances to stop resource exhaustion, especially in the IP space.

I'm not opposed to removing most of their cost from quota, but I don't
think we can do it entirely.

Michael



On Fri, Aug 19, 2016 at 3:50 AM, Tim Bell  wrote:

> I was interested to establish a consensus that
>
> - Shelved instances should not be part of the users quota
> - Quota in Glance (and associated chargeback if appropriate) is needed
>
> Glance space for us is much less expensive than people leaving their
> instances running. Equally, terminating a user’s inactive VM would not be
> popular so giving them a shelved instance would allow them to re-create it
> much more easily.
>
> Any objections to a blueprint that proposes shelving should be handled
> with the same quota model as snapshotting ?
>
> Tim
>
> On 18/08/16 19:43, "Jonathan D. Proulx"  wrote:
>
> On Thu, Aug 18, 2016 at 03:24:28PM +, Tim Bell wrote:
> :
> :We’re having a look at VM shelving for the CERN community and
> struggling to find a motivation for a private cloud user to shelve their
> instances (and free up resources they may be only using infrequently).
> :
> :The problem is that shelved instances seem to still be included in
> the user’s quota. Without internal billing, the best motivation for users
> to shelve would be to allow them to maximize the use of their quota.
> :
> :Have any other sites used shelving extensively ? How did you motivate
> your users to shelve unused resources?
>
>
> Hi Tim,
>
> We've just started looking at this and for simialar reasons.  I agree
> we should remove shelved resources from project quota. Shelved
> instances do still hold some storage resources so there may need to be
> new quota to accoutn for that some how...
>
> Currently the only motivation for our users to shelve is to get me to
> stop pestering them.
>
> We're considering policy based enforced shelving (based on some yet to
> be defined utilization metrics) but that's only a idea at this point
> not a plan.
>
> -Jon
>
>
> ___
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>



-- 
Rackspace Australia
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Shelving

[Fox, Kevin M]

+1

From: Tim Bell [tim.b...@cern.ch]
Sent: Thursday, August 18, 2016 10:50 AM
To: Jonathan D. Proulx
Cc: openstack-operators@lists.openstack.org
Subject: Re: [Openstack-operators] Shelving

I was interested to establish a consensus that

- Shelved instances should not be part of the users quota
- Quota in Glance (and associated chargeback if appropriate) is needed

Glance space for us is much less expensive than people leaving their instances 
running. Equally, terminating a user’s inactive VM would not be popular so 
giving them a shelved instance would allow them to re-create it much more 
easily.

Any objections to a blueprint that proposes shelving should be handled with the 
same quota model as snapshotting ?

Tim

On 18/08/16 19:43, "Jonathan D. Proulx"  wrote:

On Thu, Aug 18, 2016 at 03:24:28PM +, Tim Bell wrote:
:
:We’re having a look at VM shelving for the CERN community and struggling 
to find a motivation for a private cloud user to shelve their instances (and 
free up resources they may be only using infrequently).
:
:The problem is that shelved instances seem to still be included in the 
user’s quota. Without internal billing, the best motivation for users to shelve 
would be to allow them to maximize the use of their quota.
:
:Have any other sites used shelving extensively ? How did you motivate your 
users to shelve unused resources?


Hi Tim,

We've just started looking at this and for simialar reasons.  I agree
we should remove shelved resources from project quota. Shelved
instances do still hold some storage resources so there may need to be
new quota to accoutn for that some how...

Currently the only motivation for our users to shelve is to get me to
stop pestering them.

We're considering policy based enforced shelving (based on some yet to
be defined utilization metrics) but that's only a idea at this point
not a plan.

-Jon


___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Shelving

[Tim Bell]

I was interested to establish a consensus that

- Shelved instances should not be part of the users quota
- Quota in Glance (and associated chargeback if appropriate) is needed

Glance space for us is much less expensive than people leaving their instances 
running. Equally, terminating a user’s inactive VM would not be popular so 
giving them a shelved instance would allow them to re-create it much more 
easily.

Any objections to a blueprint that proposes shelving should be handled with the 
same quota model as snapshotting ?

Tim

On 18/08/16 19:43, "Jonathan D. Proulx"  wrote:

On Thu, Aug 18, 2016 at 03:24:28PM +, Tim Bell wrote:
:
:We’re having a look at VM shelving for the CERN community and struggling 
to find a motivation for a private cloud user to shelve their instances (and 
free up resources they may be only using infrequently).
:
:The problem is that shelved instances seem to still be included in the 
user’s quota. Without internal billing, the best motivation for users to shelve 
would be to allow them to maximize the use of their quota.
:
:Have any other sites used shelving extensively ? How did you motivate your 
users to shelve unused resources?


Hi Tim,

We've just started looking at this and for simialar reasons.  I agree
we should remove shelved resources from project quota. Shelved
instances do still hold some storage resources so there may need to be
new quota to accoutn for that some how...

Currently the only motivation for our users to shelve is to get me to
stop pestering them.

We're considering policy based enforced shelving (based on some yet to
be defined utilization metrics) but that's only a idea at this point
not a plan.

-Jon


___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] Shelving

[Jonathan D. Proulx]

On Thu, Aug 18, 2016 at 03:24:28PM +, Tim Bell wrote:
:
:We’re having a look at VM shelving for the CERN community and struggling to 
find a motivation for a private cloud user to shelve their instances (and free 
up resources they may be only using infrequently).
:
:The problem is that shelved instances seem to still be included in the user’s 
quota. Without internal billing, the best motivation for users to shelve would 
be to allow them to maximize the use of their quota.
:
:Have any other sites used shelving extensively ? How did you motivate your 
users to shelve unused resources?


Hi Tim,

We've just started looking at this and for simialar reasons.  I agree
we should remove shelved resources from project quota. Shelved
instances do still hold some storage resources so there may need to be
new quota to accoutn for that some how...

Currently the only motivation for our users to shelve is to get me to
stop pestering them.

We're considering policy based enforced shelving (based on some yet to
be defined utilization metrics) but that's only a idea at this point
not a plan.

-Jon

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

[Openstack-operators] Shelving

[Tim Bell]

We’re having a look at VM shelving for the CERN community and struggling to 
find a motivation for a private cloud user to shelve their instances (and free 
up resources they may be only using infrequently).

The problem is that shelved instances seem to still be included in the user’s 
quota. Without internal billing, the best motivation for users to shelve would 
be to allow them to maximize the use of their quota.

Have any other sites used shelving extensively ? How did you motivate your 
users to shelve unused resources?

Tim

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators