* Mark Mielke (mark.mie...@gmail.com) wrote: > Thank you for the work on nested virtualization. Having had live migrations > fail in the past when nested virtualization has been active, it is great to > see that clever people have been working on this problem! > > My question is about whether a migration path has been considered to allow > live migration from Qemu 2.12 hosts to Qemu 3.2 hosts, with VMX flag > enabled in the guest? > > Qemu 2.12 doesn't know about the new nested state available from newer > Linux kernels, and it might be used on a machine with an older kernel that > doesn't make the nested state available. If Qemu 3.2 is on an up-to-date > host with an up-to-date kernel that does support the nested state, I'd like > to ensure we have the ability to try the migrations. > > In the past, I've found that: > > 1) If the guest had used nested virtualization before, the migration often > fails. However, if we reboot the guest and do not use nested > virtualization, this simplifies to... > 2) If the guest has never used nested virtualization before, the migration > succeeds. > > I would like to leverage 2) as much as possible to migrate forwards to Qemu > 3.2 hosts (once it is available). I can normally enter the guest to see if > 1) is likely or not, and handle these ones specially. If only 20% of the > guests have ever used nested virtualization, then I would like the option > to safely migrate 80% of the guests using live migration, and handle the > 20% as exceptions. > > This is the 3.1 change log that got my attention: > > > - x86 machines cannot be live-migrated if nested Intel virtualization is > enabled. The next version of QEMU will be able to do live migration when > nested virtualization is enabled, if supported by the kernel. > > > I believe this is the change it refers to: > > commit d98f26073bebddcd3da0ba1b86c3a34e840c0fb8 > Author: Paolo Bonzini <pbonz...@redhat.com> > Date: Wed Nov 14 10:38:13 2018 +0100 > > target/i386: kvm: add VMX migration blocker > > Nested VMX does not support live migration yet. Add a blocker > until that is worked out. > > Nested SVM only does not support it, but unfortunately it is > enabled by default for -cpu host so we cannot really disable it. > > Signed-off-by: Paolo Bonzini <pbonz...@redhat.com> > > > This particular check seems very simplistic: > > + if ((env->features[FEAT_1_ECX] & CPUID_EXT_VMX) && !vmx_mig_blocker) { > + error_setg(&vmx_mig_blocker, > + "Nested VMX virtualization does not support live > migration yet"); > + r = migrate_add_blocker(vmx_mig_blocker, &local_err); > + if (local_err) { > + error_report_err(local_err); > + error_free(vmx_mig_blocker); > + return r; > + } > + } > + > > It fails if the flag is set, rather than if any nested virtualization has > been used before. > > I'm concerned I will end up with a requirement for *all* guests to be > restarted in order to migrate them to the new hosts, rather than just the > ones that would have a problem.
I think you should be able to migrate from 2.12->3.1 like this, but you'd hit the problem when you then try and migrate again between your new QEMUs. I guess we could modify it to wire it to machine type, so that older machine types didn't block. Dave > Thoughts? > > Thanks! > > -- > Mark Mielke <mark.mie...@gmail.com> -- Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK