On Thu, Dec 27, 2012 at 07:36:40AM -0500, Simon Grinberg wrote: > > > ----- Original Message ----- > > From: "Dan Kenigsberg" <[email protected]> > > To: "Simon Grinberg" <[email protected]> > > Cc: "arch" <[email protected]> > > Sent: Thursday, December 27, 2012 2:14:06 PM > > Subject: Re: feature suggestion: initial generation of management network > > > > On Tue, Dec 25, 2012 at 09:29:26AM -0500, Simon Grinberg wrote: > > > > > > > > > ----- Original Message ----- > > > > From: "Dan Kenigsberg" <[email protected]> > > > > To: "arch" <[email protected]> > > > > Sent: Tuesday, December 25, 2012 2:27:22 PM > > > > Subject: feature suggestion: initial generation of management > > > > network > > > > > > > > Current condition: > > > > ================== > > > > The management network, named ovirtmgmt, is created during host > > > > bootstrap. It consists of a bridge device, connected to the > > > > network > > > > device that was used to communicate with Engine (nic, bonding or > > > > vlan). > > > > It inherits its ip settings from the latter device. > > > > > > > > Why Is the Management Network Needed? > > > > ===================================== > > > > Understandably, some may ask why do we need to have a management > > > > network - why having a host with IPv4 configured on it is not > > > > enough. > > > > The answer is twofold: > > > > 1. In oVirt, a network is an abstraction of the resources > > > > required > > > > for > > > > connectivity of a host for a specific usage. This is true for > > > > the > > > > management network just as it is for VM network or a display > > > > network. > > > > The network entity is the key for adding/changing nics and IP > > > > address. > > > > 2. In many occasions (such as small setups) the management > > > > network is > > > > used as a VM/display network as well. > > > > > > > > Problems in current connectivity: > > > > ================================ > > > > According to alonbl of ovirt-host-deploy fame, and with no > > > > conflict > > > > to > > > > my own experience, creating the management network is the most > > > > fragile, > > > > error-prone step of bootstrap. > > > > > > +1, > > > I've raise that repeatedly in the past, bootstrap should not create > > > the management network but pick up the existing configuration and > > > let the engine override later with it's own configuration if it > > > differs , I'm glad that we finally get to that. > > > > > > > > > > > Currently it always creates a bridged network (even if the DC > > > > requires a > > > > non-bridged ovirtmgmt), it knows nothing about the defined MTU > > > > for > > > > ovirtmgmt, it uses ping to guess on top of which device to build > > > > (and > > > > thus requires Vdsm-to-Engine reverse connectivity), and is the > > > > sole > > > > remaining user of the addNetwork/vdsm-store-net-conf scripts. > > > > > > > > Suggested feature: > > > > ================== > > > > Bootstrap would avoid creating a management network. Instead, > > > > after > > > > bootstrapping a host, Engine would send a getVdsCaps probe to the > > > > installed host, receiving a complete picture of the network > > > > configuration on the host. Among this picture is the device that > > > > holds > > > > the host's management IP address. > > > > > > > > Engine would send setupNetwork command to generate ovirtmgmt with > > > > details devised from this picture, and according to the DC > > > > definition > > > > of > > > > ovirtmgmt. For example, if Vdsm reports: > > > > > > > > - vlan bond4.3000 has the host's IP, configured to use dhcp. > > > > - bond4 is comprises eth2 and eth3 > > > > - ovirtmgmt is defined as a VM network with MTU 9000 > > > > > > > > then Engine sends the likes of: > > > > setupNetworks(ovirtmgmt: {bridged=True, vlan=3000, iface=bond4, > > > > bonding=bond4: {eth2,eth3}, MTU=9000) > > > > > > Just one comment here, > > > In order to save time and confusion - if the ovirtmgmt is defined > > > with default values meaning the user did not bother to touch it, > > > let it pick up the VLAN configuration from the first host added in > > > the Data Center. > > > > > > Otherwise, you may override the host VLAN and loose connectivity. > > > > > > This will also solve the situation many users encounter today. > > > 1. The engine in on a host that actually has VLAN defined > > > 2. The ovirtmgmt network was not updated in the DC > > > 3. A host, with VLAN already defined is added - everything works > > > fine > > > 4. Any number of hosts are now added, again everything seems to > > > work fine. > > > > > > But, now try to use setupNetworks, and you'll find out that you > > > can't do much on the interface that contains the ovirtmgmt since > > > the definition does not match. You can't sync (Since this will > > > remove the VLAN and cause connectivity lose) you can't add more > > > networks on top since it already has non-VLAN network on top > > > according to the DC definition, etc. > > > > > > On the other hand you can't update the ovirtmgmt definition on the > > > DC since there are clusters in the DC that use the network. > > > > > > The only workaround not involving DB hack to change the VLAN on the > > > network is to: > > > 1. Create new DC > > > 2. Do not use the wizard that pops up to create your cluster. > > > 3. Modify the ovirtmgmt network to have VLANs > > > 4. Now create a cluster and add your hosts. > > > > > > If you insist on using the default DC and cluster then before > > > adding the first host, create an additional DC and move the > > > Default cluster over there. You may then change the network on the > > > Default cluster and then move the Default cluster back > > > > > > Both are ugly. And should be solved by the proposal above. > > > > > > We do something similar for the Default cluster CPU level, where we > > > set the intial level based on the first host added to the cluster. > > > > I'm not sure what Engine has for Default cluster CPU level. But I > > have > > reservation of the hysteresis in your proposal - after a host is > > added, > > the DC cannot forget ovirtmgmt's vlan. > > > > How about letting the admin edit ovirtmgmt's vlan in the DC level, > > thus > > rendering all hosts out-of-sync. The the admin could manually, or > > through a script, or in the future through a distributed operation, > > sync > > all the hosts to the definition? > > Usually if you do that you will loose connectivity to the hosts.
Yes, changing the management vlan id (or ip address) is never fun, and requires out-of-band intervention. > I'm not insisting on the automatic adjustment of the ovirtmgmt network to > match the hosts' (that is just a nice touch) we can take the allow edit > approach. > > But allow to change VLAN on the ovirtmgmt network will indeed solve the issue > I'm trying to solve while creating another issue of user expecting that we'll > be able to re-tag the host from the engine side, which is challenging to do. > > On the other hand, if we allow to change the VLAN as long as the change > matches the hosts' configuration, it will both solve the issue while not > eluding the user to think that we really can solve the chicken and egg issue > of re-tag the entire system. > > Now with the above ability you do get a flow to do the re-tag. > 1. Place all the hosts in maintenance > 2. Re-tag the ovirtmgmt on all the hosts > 3. Re-tag the hosts on which the engine on > 4. Activate the hosts - this should work well now since connectivity exist > 5. Change the tag on ovirtmgmt on the engine to match the hosts' > > Simple and clear process. > > When the workaround of creating another DC was not possible since the system > was already long in use and the need was re-tag of the network the above is > what I've recommended in the, except that steps 4-5 where done as: > 4. Stop the engine > 5. Change the tag in the DB > 6. Start the engine > 7. Activate the hosts Sounds reasonable to me - but as far as I am aware this is not tightly related to the $Subject, which is the post-boot ovirtmgmt definition. I've added a few details to http://www.ovirt.org/Features/Normalized_ovirtmgmt_Initialization#Engine and I would apreciate a review from someone with intimate Engine know-how. Dan. _______________________________________________ Arch mailing list [email protected] http://lists.ovirt.org/mailman/listinfo/arch
