For a guest domain defined with a large number of macvtap devices, it takes an
exceedingly long time to boot the guest. In a test of a guest domain configured
with 82 macvtap devices, it took over two minutes for the guest to boot. An
strace of the ioctl calls during guest start up showed the SIOCGIFFLAGS ioctl
literally being invoked 3,403 times. I was able to isolate the source of the
ioctl calls to the*virNetDevMacVLanCreateWithVPortProfile* function
in*virnetdevmacvlan.c*. The macvtap interface name is created by looping over a
counter variable, starting with zero, and appending the counter value to
'macvtap'. With each iteration, a call is made to*virNetDevExists*
(SIOCGIFFLAGS ioctl) to determine if a device with that name already exists,
until a unique name is created. In the test case cited above, to create an
interface name for the 82nd macvtap device, the*virNetDevExists* function will
be called for interface names 'macvtap0' to 'macvtap80' before it is determined
that 'mavtap81' can be used. So if N is the number of macvtap interfaces
defined for a guest, the SIOCGIFFLAGS ioctl will be invoked (N x N + N)/2 times
to find an unused macvtap device names. That's assuming only one guest is being
started, who knows how many times the ioctl may have to be called in an
installation running a large number of guests defined with macvtap devices.
I was able to reduce the amount of time for starting a guest domain defined
with 82 macvtap devices from over 2 minutes to about 14 seconds by keeping
track of the interface name suffixes previously used. I defined two static bit
maps (virBitmap), one each for macvtap and macvlan device name suffixes. When a
macvtap/macvlan device is created, the index of the next clear bit
(virBitmapNextClearBit) is retrieved to create the name. If an interface with
that name does not exist, the device is created and the bit at the index used
to create the interface name is set (virBitmapSetBit). When a macvtap/macvlan
device is deleted, if the interface name has the pattern 'macvtap%d' or
'macvlan%d', the suffix is parsed into a bit index and used to clear the
(virBitMapClearBit) bit in the respective bitmap.
I am not sure that is the best design because there is no way to track
interface names used to create macvtap devices outside of libvirt, for example
using the ip command. There may also be other issues I've not contemplated. I
included a couple of additional ideas below and am looking for comments or
other suggestions that I have not considered.
* Define a global counter variable initialized to 0, that gets
incremented each time an interface name is created, to keep track of
the last used interface name suffix. At some maximum value, the
counter will be set back to 0.
* Append a random number to 'macvlan' or 'macvtap' when creating the
interface name. Of course, the number of digits would have to be
limited so the interface name would not exceed the maximum allowed.
* Create the interface name in code that has more knowledge of the
environment and pass the name into the
*virNetDevMacVLanCreateWithVPortProfile* function via the *tgifname*
parameter. For example, the *qemuBuildCommandLine* function in
*qemu_command.c* contains the loop that iterates over the network
devices defined for the guest domain that ultimately get created via
the *virNetDevMacVLanCreateWithVPortProfile* function. That function
has access to the network device configuration and at the very least
could ensure none of the names previously defined for the guest
aren't used. I believe it would be matter of creating a macvtap
interface name - e.g., maybe a call to some function in
*virnetdevmacvlan.c* - and setting the name in the virDomainNetDef
structure prior to invoking *qemuBuildInterfaceCommandLine*?
There are shortcomings in all of these ideas, so if you have a better
one, feel free to present it.
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
