Re: VMX ideas
On Mon, Nov 03, 2008 at 08:24:49AM +0100, Alexander Graf wrote: > > > Am 03.11.2008 um 00:36 schrieb "Glauber Costa" <[EMAIL PROTECTED]>: > > >On Sun, Nov 2, 2008 at 2:18 PM, Avi Kivity <[EMAIL PROTECTED]> wrote: > >>Avi Kivity wrote: > >>> > >>>Daniel P. Berrange wrote: > > Yes, on Fedora, if you install the 'kvm' userspace package, we > drop a > script into /etc/sysconfig/modules/kvm.modules that cause kvm- > intel.ko > or kvm-amd.ko to be loaded at boot time. > > >>> > >>>Does the postinstall also load the modules? If not, users who don't > >>>reboot are in for an unpleasant surprise. > >>> > >> > >>I checked and it doesn't. > >Funny. Rawhide version does exactly that. > > So when a Fedora user installs the kvm userspace, he can't run vmware/ > parallels/vbox and the like without fiddling? Fedora focuses on making the best possible free software experiance, without compromising it for sake of closed source software. If vendors of out of tree kernel modules want a good experiance for their users, they can work to get the neccessary support merged in upstream kernel trees. Many vendors over the years have shipped out of tree network or disk drivers - we don't avoid loading the in kernel driver just in case in conflicts with a out of tree replacement nic/disk module. The same applies for virtualization - one in tree kernel module should be made to be sufficient for all userspace apps that want to use virt. KVM is the only in tree virt module at this time, so we load it by default. If the KVM module isn't sufficient to other userspace virt apps, then they need to work with the kernel community to address that, not create many out of tree modules duplicating each other's functionality. Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :| -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: VMX ideas
Am 03.11.2008 um 00:36 schrieb "Glauber Costa" <[EMAIL PROTECTED]>: On Sun, Nov 2, 2008 at 2:18 PM, Avi Kivity <[EMAIL PROTECTED]> wrote: Avi Kivity wrote: Daniel P. Berrange wrote: Yes, on Fedora, if you install the 'kvm' userspace package, we drop a script into /etc/sysconfig/modules/kvm.modules that cause kvm- intel.ko or kvm-amd.ko to be loaded at boot time. Does the postinstall also load the modules? If not, users who don't reboot are in for an unpleasant surprise. I checked and it doesn't. Funny. Rawhide version does exactly that. So when a Fedora user installs the kvm userspace, he can't run vmware/ parallels/vbox and the like without fiddling? That doesn't exactly sound like a good user experience to me. I'll try to post an RFC for 2. later today. Alex -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html -- Glauber Costa. "Free as in Freedom" http://glommer.net "The less confident you are, the more serious you have to act." -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: VMX ideas
Am 02.11.2008 um 11:28 schrieb Avi Kivity <[EMAIL PROTECTED]>: Gerd Hoffmann wrote: 3. Create a root mode bloat framework 4. Enter vmx root mode when the first user opens /dev/kvm and leave it when nobody uses it any more? Also note that there has been some discussion on lkml to move vmx- root on/off out of the kvm module into the kernel. There are a few places where it would be useful for the kernel to handle that without the kvm module being involved, crashdump kexec for example. With that and some locking in place (so only one can use vmx-root at a time) and hypervisors being friendly to each other and grab vmx-root only when they have a guest to run we should be set That means that although kvm is loaded it may be unable to service / dev/kvm... something I'd like to avoid. This would happen even with a vmx framework in place - unless we really always activate root mode, which would break 32-bit vmware/vbox. The way I have it implemented atm, root mode is activated on first usage. If kvm happens to be blocked out of using vmx due to various reasons, create_vcpu would fail. What exactly is bad about this? And even if we were to always activate root mode, a vmx user that needs exclusive access (e.g. uses vpids) could still block kvm. Alex -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: VMX ideas
On Sun, Nov 2, 2008 at 2:18 PM, Avi Kivity <[EMAIL PROTECTED]> wrote: > Avi Kivity wrote: >> >> Daniel P. Berrange wrote: >>> >>> Yes, on Fedora, if you install the 'kvm' userspace package, we drop a >>> script into /etc/sysconfig/modules/kvm.modules that cause kvm-intel.ko >>> or kvm-amd.ko to be loaded at boot time. >>> >> >> Does the postinstall also load the modules? If not, users who don't >> reboot are in for an unpleasant surprise. >> > > I checked and it doesn't. Funny. Rawhide version does exactly that. > > -- > error compiling committee.c: too many arguments to function > > -- > To unsubscribe from this list: send the line "unsubscribe kvm" in > the body of a message to [EMAIL PROTECTED] > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- Glauber Costa. "Free as in Freedom" http://glommer.net "The less confident you are, the more serious you have to act." -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: VMX ideas
Avi Kivity wrote: Daniel P. Berrange wrote: Yes, on Fedora, if you install the 'kvm' userspace package, we drop a script into /etc/sysconfig/modules/kvm.modules that cause kvm-intel.ko or kvm-amd.ko to be loaded at boot time. Does the postinstall also load the modules? If not, users who don't reboot are in for an unpleasant surprise. I checked and it doesn't. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: VMX ideas
Daniel P. Berrange wrote: Yes, on Fedora, if you install the 'kvm' userspace package, we drop a script into /etc/sysconfig/modules/kvm.modules that cause kvm-intel.ko or kvm-amd.ko to be loaded at boot time. Does the postinstall also load the modules? If not, users who don't reboot are in for an unpleasant surprise. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: VMX ideas
On Sun, Nov 02, 2008 at 12:26:00PM +0200, Avi Kivity wrote: > > Please copy [EMAIL PROTECTED] and > [EMAIL PROTECTED] to get feedback. > > (on the distro I use (F9), kvm is auto-loaded. or did I change some > config script and forget?) Yes, on Fedora, if you install the 'kvm' userspace package, we drop a script into /etc/sysconfig/modules/kvm.modules that cause kvm-intel.ko or kvm-amd.ko to be loaded at boot time. Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :| -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: VMX ideas
Gerd Hoffmann wrote: 3. Create a root mode bloat framework 4. Enter vmx root mode when the first user opens /dev/kvm and leave it when nobody uses it any more? Also note that there has been some discussion on lkml to move vmx-root on/off out of the kvm module into the kernel. There are a few places where it would be useful for the kernel to handle that without the kvm module being involved, crashdump kexec for example. With that and some locking in place (so only one can use vmx-root at a time) and hypervisors being friendly to each other and grab vmx-root only when they have a guest to run we should be set That means that although kvm is loaded it may be unable to service /dev/kvm... something I'd like to avoid. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: VMX ideas
Alexander Graf wrote: Hi list, I really do like KVM and would love to make using it a complete no-brainer user experience. For the average user, loading a kernel module (and unloading it) isn't really one of the most common tasks. So I was considering to automatically load kvm-intel and kvm-amd on bootup, when the CPU has a CPUID flag. Unfortunately it doesn't work that easily. VMX enters Root mode, setting a page to use for the VMCS. KVM sets up this page when the kvm-intel module gets loaded. This means, as soon as kvm-intel is modprobe'd, vmware, virtual box, parallels and the like are screwed. From a distribution perspective, this isn't exactly an ideal situation. So I was thinking hard on what to do to circumvent this problem and came up with several approaches: 1. Export some functions that could be used by 3rd party hypervisors to "occupy" the VMCS region This would mean, that the others' kernel modules have dependencies on KVM. I don't think that's really desirable. Furthermore it wouldn't work with kvm compiled as external module. 2. Create a root mode base framework This is the Mac OS X approach. They have a small piece of code in the kernel, that sets up the root mode, handles suspend/resume and maintains a ref counter on its usage. Since the root mode is set on bootup, all hypervisors would need to make use of that framework. While I like that approach for its simplicity, it does not allow for parallel execution of VMs with different hypervisors. Is this a real problem? 3. Create a root mode bloat framework Taking approach 2. we could also export functions to handle VPID management and VMXPTRLD, which would make it possible for multiple hypervisors to run concurrently. This is the most elegant approach IMHO, but also needs the most maintenance, as new revisions of VMX could possibly break the whole concept. I started working on 2. and wanted to put in some parts of 3., realizing that it's more work and code change than I'd like. Any comments on these ideas? I'd really love to have KVM always loaded somehow. I like 2 as well. We already started moving in this direction in order to get kexec working well with kvm. Please copy [EMAIL PROTECTED] and [EMAIL PROTECTED] to get feedback. (on the distro I use (F9), kvm is auto-loaded. or did I change some config script and forget?) -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: VMX ideas
On 29.10.2008, at 16:36, Gerd Hoffmann wrote: Alexander Graf wrote: So I was thinking hard on what to do to circumvent this problem and came up with several approaches: 3. Create a root mode bloat framework 4. Enter vmx root mode when the first user opens /dev/kvm and leave it when nobody uses it any more? Also note that there has been some discussion on lkml to move vmx-root on/off out of the kvm module into the kernel. There are a few places where it would be useful for the kernel to handle that without the kvm module being involved, crashdump kexec for example. With that and some locking in place (so only one can use vmx-root at a time) and hypervisors being friendly to each other and grab vmx-root only when they have a guest to run we should be set? If we simply enter/exit VT root mode on kvm starts/stops every single hypervisor would still have to maintain all the interaction bits itself, like "Is this CPU VT capable?", "May I run VT here?", hotplug events, suspend/resume, reboot, etc. It would help to have one place to keep all of that, so suspend doesn't magically break when using VMware vs. KVM. That's basically the reason I didn't even propose 4. :-). If you think that's a good solution, feel free to convince me. Alex -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: VMX ideas
On 29.10.2008, at 16:45, Anthony Liguori wrote: Gerd Hoffmann wrote: Alexander Graf wrote: So I was thinking hard on what to do to circumvent this problem and came up with several approaches: 3. Create a root mode bloat framework 4. Enter vmx root mode when the first user opens /dev/kvm and leave it when nobody uses it any more? Also note that there has been some discussion on lkml to move vmx- root on/off out of the kvm module into the kernel. There are a few places where it would be useful for the kernel to handle that without the kvm module being involved, crashdump kexec for example. With that and some locking in place (so only one can use vmx-root at a time) and hypervisors being friendly to each other and grab vmx-root only when they have a guest to run we should be set? There was some patch that went into KVM that would indicate in CPU state that VT root mode was enabled so that VMware could detect that someone else was using VT. Look through the archives. The only patch I've found is that KVM sets CR4.VMXE to 0 on unload (hardware_disable). This doesn't really help, as my goal is to have the module loaded at boot-time. Alex -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: VMX ideas
Gerd Hoffmann wrote: Alexander Graf wrote: So I was thinking hard on what to do to circumvent this problem and came up with several approaches: 3. Create a root mode bloat framework 4. Enter vmx root mode when the first user opens /dev/kvm and leave it when nobody uses it any more? Also note that there has been some discussion on lkml to move vmx-root on/off out of the kvm module into the kernel. There are a few places where it would be useful for the kernel to handle that without the kvm module being involved, crashdump kexec for example. With that and some locking in place (so only one can use vmx-root at a time) and hypervisors being friendly to each other and grab vmx-root only when they have a guest to run we should be set? There was some patch that went into KVM that would indicate in CPU state that VT root mode was enabled so that VMware could detect that someone else was using VT. Look through the archives. Regards, Anthony Liguori cheers, Gerd -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: VMX ideas
Alexander Graf wrote: > So I was thinking hard on what to do to circumvent this problem and came > up with several approaches: > 3. Create a root mode bloat framework 4. Enter vmx root mode when the first user opens /dev/kvm and leave it when nobody uses it any more? Also note that there has been some discussion on lkml to move vmx-root on/off out of the kvm module into the kernel. There are a few places where it would be useful for the kernel to handle that without the kvm module being involved, crashdump kexec for example. With that and some locking in place (so only one can use vmx-root at a time) and hypervisors being friendly to each other and grab vmx-root only when they have a guest to run we should be set? cheers, Gerd -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: VMX ideas
On 29.10.2008, at 16:01, Joerg Roedel wrote: On Wed, Oct 29, 2008 at 03:52:21PM +0100, Alexander Graf wrote: Hi list, I really do like KVM and would love to make using it a complete no- brainer user experience. For the average user, loading a kernel module (and unloading it) isn't really one of the most common tasks. So I was considering to automatically load kvm-intel and kvm-amd on bootup, when the CPU has a CPUID flag. Unfortunately it doesn't work that easily. VMX enters Root mode, setting a page to use for the VMCS. KVM sets up this page when the kvm-intel module gets loaded. This means, as soon as kvm-intel is modprobe'd, vmware, virtual box, parallels and the like are screwed. From a distribution perspective, this isn't exactly an ideal situation. So I was thinking hard on what to do to circumvent this problem and came up with several approaches: If you want to allow parallel execution of different hypervisors at the same time you have a similar problem with SVM because you don't know which hypervisor uses which ASID. Yes, while writing the mail I was thinking to bring that on too, but figured in the end that the mail's title is "VMX Ideas", so I didn't ;-). Of course, if we want to go with 3., we'd need to write a common framework for the ASIDs in SVM that could handle activation of SVM for us too. Right now the situation on VMX is a lot worse than on SVM though, as loading the module basically blocks every other hypervisor. Alex -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: VMX ideas
On Wed, Oct 29, 2008 at 03:52:21PM +0100, Alexander Graf wrote: > Hi list, > > I really do like KVM and would love to make using it a complete no- > brainer user experience. For the average user, loading a kernel module > (and unloading it) isn't really one of the most common tasks. > > So I was considering to automatically load kvm-intel and kvm-amd on > bootup, when the CPU has a CPUID flag. Unfortunately it doesn't work > that easily. VMX enters Root mode, setting a page to use for the VMCS. > KVM sets up this page when the kvm-intel module gets loaded. This > means, as soon as kvm-intel is modprobe'd, vmware, virtual box, > parallels and the like are screwed. > From a distribution perspective, this isn't exactly an ideal situation. > > So I was thinking hard on what to do to circumvent this problem and > came up with several approaches: If you want to allow parallel execution of different hypervisors at the same time you have a similar problem with SVM because you don't know which hypervisor uses which ASID. Joerg -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
