Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

There is now a blueprint [1] and draft spec [2]. Reviews welcomed. [1] https://blueprints.launchpad.net/nova/+spec/reshape-provider-tree [2] https://review.openstack.org/#/c/572583/ On 06/04/2018 06:00 PM, Eric Fried wrote: > There has been much discussion. We've gotten to a point of an

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

There has been much discussion. We've gotten to a point of an initial proposal and are ready for more (hopefully smaller, hopefully conclusive) discussion. To that end, there will be a HANGOUT tomorrow (TUESDAY, JUNE 5TH) at 1500 UTC. Be in #openstack-placement to get the link to join. The

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

> FWIW, I don't have a problem with the virt driver "knowing about > allocations". What I have a problem with is the virt driver *claiming > resources for an instance*. +1000. > That's what the whole placement claims resources things was all about, > and I'm not interested in stepping back to

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

On 06/01/2018 03:02 PM, Dan Smith wrote: Dan, you are leaving out the parts of my response where I am agreeing with you and saying that your "Option #2" is probably the things we should go with. No, what you said was: I would vote for Option #2 if it comes down to it. Implying (to me at

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

> Dan, you are leaving out the parts of my response where I am agreeing > with you and saying that your "Option #2" is probably the things we > should go with. No, what you said was: >> I would vote for Option #2 if it comes down to it. Implying (to me at least) that you still weren't in favor

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

On 05/31/2018 02:26 PM, Eric Fried wrote: 1. Make everything perform the pivot on compute node start (which can be re-used by a CLI tool for the offline case) 2. Make everything default to non-nested inventory at first, and provide a way to migrate a compute node and its instances one at

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

Dan, you are leaving out the parts of my response where I am agreeing with you and saying that your "Option #2" is probably the things we should go with. -jay On 06/01/2018 12:22 PM, Dan Smith wrote: So, you're saying the normal process is to try upgrading the Linux kernel and associated

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

> So, you're saying the normal process is to try upgrading the Linux > kernel and associated low-level libs, wait the requisite amount of > time that takes (can be a long time) and just hope that everything > comes back OK? That doesn't sound like any upgrade I've ever seen. I'm saying I think

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

Sylvain- On 05/31/2018 02:41 PM, Sylvain Bauza wrote: > > > On Thu, May 31, 2018 at 8:26 PM, Eric Fried > wrote: > > > 1. Make everything perform the pivot on compute node start (which can be > >    re-used by a CLI tool for the offline case) > > 2. Make

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

On Thu, May 31, 2018 at 8:26 PM, Eric Fried wrote: > > 1. Make everything perform the pivot on compute node start (which can be > >re-used by a CLI tool for the offline case) > > 2. Make everything default to non-nested inventory at first, and provide > >a way to migrate a compute node

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

On Thu, May 31, 2018 at 7:44 PM, Chris Dent wrote: > On Thu, 31 May 2018, Dan Smith wrote: > > I kinda think we need to either: >> >> 1. Make everything perform the pivot on compute node start (which can be >> re-used by a CLI tool for the offline case) >> > > This sounds effectively like:

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

On 05/31/2018 01:09 PM, Dan Smith wrote: My feeling is that we should not attempt to "migrate" any allocations or inventories between root or child providers within a compute node, period. While I agree this is the simplest approach, it does put a lot of responsibility on the operators to do

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

On Thu, May 31, 2018 at 7:09 PM, Dan Smith wrote: > > My feeling is that we should not attempt to "migrate" any allocations > > or inventories between root or child providers within a compute node, > > period. > > While I agree this is the simplest approach, it does put a lot of > responsibility

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

On Thu, May 31, 2018 at 5:00 PM, Jay Pipes wrote: > On 05/31/2018 05:10 AM, Sylvain Bauza wrote: > >> After considering the whole approach, discussing with a couple of folks >> over IRC, here is what I feel the best approach for a seamless upgrade : >> - VGPU inventory will be kept on root RP

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

On Thu, May 31, 2018 at 4:34 PM, Jay Pipes wrote: > On 05/29/2018 09:12 AM, Sylvain Bauza wrote: > >> We could keep the old inventory in the root RP for the previous vGPU type >> already supported in Queens and just add other inventories for other vGPU >> types now supported. That looks possibly

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

Chris- >> virt driver isn't supposed to talk to placement directly, or know >> anything about allocations? > > For sake of discussion, how much (if any) easier would it be if we > got rid of this restriction? At this point, having implemented the update_[from_]provider_tree flow as we have, it

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

On Thu, May 31, 2018 at 3:54 PM, Eric Fried wrote: > This seems reasonable, but... > > On 05/31/2018 04:34 AM, Balázs Gibizer wrote: > > > > > > On Thu, May 31, 2018 at 11:10 AM, Sylvain Bauza > wrote: > >>> > >> > >> After considering the whole approach, discussing with a couple of > >> folks

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

On Thu, 31 May 2018, Eric Fried wrote: But how would this be accomplished, in light of the current "separation of responsibilities" drawn at the virt driver interface, whereby the virt driver isn't supposed to talk to placement directly, or know anything about allocations? Here's a first pass:

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

Rats, typo correction below. On 05/31/2018 01:26 PM, Eric Fried wrote: >> 1. Make everything perform the pivot on compute node start (which can be >>re-used by a CLI tool for the offline case) >> 2. Make everything default to non-nested inventory at first, and provide >>a way to migrate a

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

> 1. Make everything perform the pivot on compute node start (which can be >re-used by a CLI tool for the offline case) > 2. Make everything default to non-nested inventory at first, and provide >a way to migrate a compute node and its instances one at a time (in >place) to roll

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

On Thu, 31 May 2018, Dan Smith wrote: I kinda think we need to either: 1. Make everything perform the pivot on compute node start (which can be re-used by a CLI tool for the offline case) This sounds effectively like: validate my inventory and allocations at compute node start, correcting

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

> My feeling is that we should not attempt to "migrate" any allocations > or inventories between root or child providers within a compute node, > period. While I agree this is the simplest approach, it does put a lot of responsibility on the operators to do work to sidestep this issue, which

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

On 05/31/2018 05:10 AM, Sylvain Bauza wrote: After considering the whole approach, discussing with a couple of folks over IRC, here is what I feel the best approach for a seamless upgrade :  - VGPU inventory will be kept on root RP (for the first type) in Queens so that a compute service

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

On 05/30/2018 07:06 AM, Balázs Gibizer wrote: The nova-manage is another possible way similar to my idea #c) but there I imagined the logic in placement-manage instead of nova-manage. Please note there is no placement-manage CLI tool. Best, -jay

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

On 05/29/2018 09:12 AM, Sylvain Bauza wrote: We could keep the old inventory in the root RP for the previous vGPU type already supported in Queens and just add other inventories for other vGPU types now supported. That looks possibly the simpliest option as the virt driver knows that. What

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

questions) Subject: Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers This seems reasonable, but... On 05/31/2018 04:34 AM, Balázs Gibizer wrote: > > > On Thu, May 31, 2018 at 11:10 AM, Sylvain Bauza wrote: >>> >> >> After

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

This seems reasonable, but... On 05/31/2018 04:34 AM, Balázs Gibizer wrote: > > > On Thu, May 31, 2018 at 11:10 AM, Sylvain Bauza wrote: >>> >> >> After considering the whole approach, discussing with a couple of >> folks over IRC, here is what I feel the best approach for a seamless >>

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

On Thu, May 31, 2018 at 11:10 AM, Sylvain Bauza wrote: After considering the whole approach, discussing with a couple of folks over IRC, here is what I feel the best approach for a seamless upgrade : - VGPU inventory will be kept on root RP (for the first type) in Queens so that a

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

On Wed, May 30, 2018 at 1:06 PM, Balázs Gibizer wrote: > > > On Tue, May 29, 2018 at 3:12 PM, Sylvain Bauza wrote: > >> >> >> On Tue, May 29, 2018 at 2:21 PM, Balázs Gibizer < >> balazs.gibi...@ericsson.com> wrote: >> >>> >>> >>> On Tue, May 29, 2018 at 1:47 PM, Sylvain Bauza >>> wrote: >>>

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

On Tue, May 29, 2018 at 3:12 PM, Sylvain Bauza wrote: On Tue, May 29, 2018 at 2:21 PM, Balázs Gibizer wrote: On Tue, May 29, 2018 at 1:47 PM, Sylvain Bauza wrote: Le mar. 29 mai 2018 à 11:02, Balázs Gibizer a écrit : On Tue, May 29, 2018 at 9:38 AM, Sylvain Bauza wrote: > >

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

On Tue, May 29, 2018 at 2:21 PM, Balázs Gibizer wrote: > > > On Tue, May 29, 2018 at 1:47 PM, Sylvain Bauza wrote: > >> >> >> Le mar. 29 mai 2018 à 11:02, Balázs Gibizer >> a écrit : >> >>> >>> >>> On Tue, May 29, 2018 at 9:38 AM, Sylvain Bauza >>> wrote: >>> > >>> > >>> > On Tue, May 29,

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

On Tue, May 29, 2018 at 1:47 PM, Sylvain Bauza wrote: Le mar. 29 mai 2018 à 11:02, Balázs Gibizer a écrit : On Tue, May 29, 2018 at 9:38 AM, Sylvain Bauza wrote: > > > On Tue, May 29, 2018 at 3:08 AM, TETSURO NAKAMURA > wrote > >> > In that situation, say for example with VGPU

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

Le mar. 29 mai 2018 à 11:02, Balázs Gibizer a écrit : > > > On Tue, May 29, 2018 at 9:38 AM, Sylvain Bauza > wrote: > > > > > > On Tue, May 29, 2018 at 3:08 AM, TETSURO NAKAMURA > > wrote > > > >> > In that situation, say for example with VGPU inventories, that > >> would mean > >> > that the

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

On Tue, May 29, 2018 at 11:52 AM, Sylvain Bauza wrote: 2018-05-29 11:01 GMT+02:00 Balázs Gibizer : On Tue, May 29, 2018 at 9:38 AM, Sylvain Bauza wrote: On Tue, May 29, 2018 at 3:08 AM, TETSURO NAKAMURA wrote > In that situation, say for example with VGPU inventories, that

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

2018-05-29 11:01 GMT+02:00 Balázs Gibizer : > > > On Tue, May 29, 2018 at 9:38 AM, Sylvain Bauza wrote: > >> >> >> On Tue, May 29, 2018 at 3:08 AM, TETSURO NAKAMURA < >> nakamura.tets...@lab.ntt.co.jp> wrote >> >> > In that situation, say for example with VGPU inventories, that would >>> mean

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

On Tue, May 29, 2018 at 9:38 AM, Sylvain Bauza wrote: On Tue, May 29, 2018 at 3:08 AM, TETSURO NAKAMURA wrote > In that situation, say for example with VGPU inventories, that would mean > that the compute node would stop reporting inventories for its root RP, but > would rather

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

On Tue, May 29, 2018 at 3:08 AM, TETSURO NAKAMURA < nakamura.tets...@lab.ntt.co.jp> wrote: > Hi, > > > Do I still understand correctly ? If yes, perfect, let's jump to my > upgrade > > concern. > > Yes, I think. The old microversions look into only root providers and give > up providing resources

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

Hi, > Do I still understand correctly ? If yes, perfect, let's jump to my upgrade > concern. Yes, I think. The old microversions look into only root providers and give up providing resources if a root provider itself doesn't have enough inventories for requested resources. But the new

[openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

Hi, I already told about that in a separate thread, but let's put it here too for more visibility. tl;dr: I suspect existing allocations are being lost when we upgrade a compute service from Queens to Rocky, if those allocations are made against inventories that are now provided by a child