On 14:40 Thu 27 Mar , Apollon Oikonomopoulos wrote: > On 13:16 Thu 27 Mar , Thomas Thrainer wrote: > > Hi Apollon, > > > > Thanks for the in-depth analysis! Please see my comments inline: > > > > > > On Thu, Mar 27, 2014 at 12:04 PM, Apollon Oikonomopoulos > > <[email protected]>wrote: > > > The downside of this is that all instances will see all disk and NIC > > > PCI > > > slots changed on the first boot with the new model (although relative > > > order will be preserved). Linux guests will cope with this fine, but I'm > > > not sure how Windows guests will do. > > > > > > > I think the same holds true for normal reboots right now: on the master > > node, we can't tell in which PCI slot the disk actually ended up. So after > > stopping the instance, a (re-)start will put the disks in the PCI slots in > > the order in which they are in the cluster configuration, not in the order > > they were when they were first hotplugged. Or am I wrong here? Also, I'm > > not sure if right now the relative order is preserved, as you can come up > > with an example like the above for the current implementation as well. > > In this regard, the above solution is not better, as the PCI slot > > assignment is not put into the master configuration neither. > > That's true. Right now everything works, because even if devices are in > the "wrong" order on the PCI bus because of "holes" created after boot, > the order in which they appeared to the OS is actually correct (eth1 > appeared after - in terms of time - eth0 because of hotplugging, vdb > after vda, etc). A hypervisor reboot will reassign the PCI slots and > they will be correct in that respect, but I have a suspicion that a > reboot from within the instance right after hotplugging in a "hole" will > get the disks/NICs in the wrong order. This doesn't really matter for > NICs (since they are identified by MAC address most of the time), but it > matters for disks if you're not using UUIDs.
And to clarify things a bit, such a behavior can only be avoided either if we hardcode slots in the cluster config, or if we treat disks and NICs not as stacks, but as arrays with fixed size and possibly empty positions. i.e. hot-removing NIC 1 will leave slot 1 empty, instead of shifting all NICs by one. In that way we could map NICs and disks to PCI slots persistently, but this is a design change in ganeti. /A
