[Xen-devel] PVHVM drivers in upstream linux kernel
Hi, looking into the upstream linux sources I realized that the PVHVM drivers of XEN are only available with the pvops kernel. Is this on purpose? Shouldn't the frontend drivers, xen/platform-pci.c etc. be configurable without having to enable CONFIG_PARAVIRT? Juergen ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] PVHVM drivers in upstream linux kernel
On 12/02/2014 11:54 AM, David Vrabel wrote: On 02/12/14 09:39, Juergen Gross wrote: Hi, looking into the upstream linux sources I realized that the PVHVM drivers of XEN are only available with the pvops kernel. Is this on purpose? Shouldn't the frontend drivers, xen/platform-pci.c etc. be configurable without having to enable CONFIG_PARAVIRT? I suppose that would be possible but I don't think it's a useful configuration because you would lose PV spinlocks for example. Having no pv spinlocks on a single vcpu HVM guest isn't so bad. Having no pv-drivers for disks and networking is worse. Juergen ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] PVHVM drivers in upstream linux kernel
On Tue, 2014-12-02 at 10:54 +, David Vrabel wrote: On 02/12/14 09:39, Juergen Gross wrote: Hi, looking into the upstream linux sources I realized that the PVHVM drivers of XEN are only available with the pvops kernel. Is this on purpose? Shouldn't the frontend drivers, xen/platform-pci.c etc. be configurable without having to enable CONFIG_PARAVIRT? I suppose that would be possible but I don't think it's a useful configuration because you would lose PV spinlocks for example. IIRC the reason this hasn't been implemented until now is that refactoring would be required to the various bits of driver code which assumes PAE + PARAVIRT when they aren't strictly needed, e.g. grant table code. Whether its worth the churn at this stage is debatable, but I think the (in)ability to use PV spinlocks is a red-herring. Adding PV drivers to an HVM guest is a useful thing to do, even without PV spinlocks. PV IO gets you far more incremental benefit than the locks do, adding PV IO paths is the number 1 thing which should be done to any guest. One actual usecase is installing from a distro installer which isn't PAE, let alone PARAVIRT enabled[0], to get far enough that you can install a more capable PVHVM kernel with more bells and whistles. If there were distros around who refused wholesale to enable PARAVIRT even in a non-default kernel then it would be more likely that they could be convinced to enable a set of PV IO drivers, since they have 0 impact on a non-PARAVIRT system, and still give significant benefit to Xen users. I don't know of any of the major distros are refusing PARAVIRT in this way though. Ian. [0] The default i386 Debian installer falls into this camp, but you can use the special PV Xen variant to install as PVHVM too so it's not so critical. ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] PVHVM drivers in upstream linux kernel
On 12/02/2014 12:05 PM, Ian Campbell wrote: On Tue, 2014-12-02 at 10:54 +, David Vrabel wrote: On 02/12/14 09:39, Juergen Gross wrote: Hi, looking into the upstream linux sources I realized that the PVHVM drivers of XEN are only available with the pvops kernel. Is this on purpose? Shouldn't the frontend drivers, xen/platform-pci.c etc. be configurable without having to enable CONFIG_PARAVIRT? I suppose that would be possible but I don't think it's a useful configuration because you would lose PV spinlocks for example. IIRC the reason this hasn't been implemented until now is that refactoring would be required to the various bits of driver code which assumes PAE + PARAVIRT when they aren't strictly needed, e.g. grant table code. Whether its worth the churn at this stage is debatable, but I think the (in)ability to use PV spinlocks is a red-herring. Adding PV drivers to an HVM guest is a useful thing to do, even without PV spinlocks. PV IO gets you far more incremental benefit than the locks do, adding PV IO paths is the number 1 thing which should be done to any guest. I take this as an ack to change this. :-) One actual usecase is installing from a distro installer which isn't PAE, let alone PARAVIRT enabled[0], to get far enough that you can install a more capable PVHVM kernel with more bells and whistles. If there were distros around who refused wholesale to enable PARAVIRT even in a non-default kernel then it would be more likely that they could be convinced to enable a set of PV IO drivers, since they have 0 impact on a non-PARAVIRT system, and still give significant benefit to Xen users. I don't know of any of the major distros are refusing PARAVIRT in this way though. I think we have customers wanting to run a default kernel as domU. So it isn't always the distro refusing paravirt, it might be the user... Okay, how do the current config settings regarding Xen look like? We have: - XEN depending on PARAVIRT - XEN_DOM0 depending on XEN and others - XEN_BACKEND depending on XEN_DOM0 - various backend drivers depending on XEN_BACKEND - XEN_PVHVM depending on XEN - various frontend drivers depending on XEN (even if some are not depending on XEN according to Kconfig, they do as the complete drivers/xen directory is made only if CONFIG_XEN is defined) To sort things out I'd suggest to: - make XEN independent from PARAVIRT - let XEN_DOM0 select XEN_BACKEND, PARAVIRT, XEN - let XEN_BACKEND select PARAVIRT, XEN (I'd like to be able to build a driver domain without XEN_DOM0) - introduce XEN_FRONTEND, let it select XEN - let frontend drivers and drivers needed by those depend on XEN_FRONTEND - let XEN_PVHVM select XEN_FRONTEND - don't skip drivers/xen on make, as XEN might be selected via a config item in that directory Juergen ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] PVHVM drivers in upstream linux kernel
On Tue, 2014-12-02 at 12:33 +0100, Juergen Gross wrote: On 12/02/2014 12:05 PM, Ian Campbell wrote: On Tue, 2014-12-02 at 10:54 +, David Vrabel wrote: On 02/12/14 09:39, Juergen Gross wrote: Hi, looking into the upstream linux sources I realized that the PVHVM drivers of XEN are only available with the pvops kernel. Is this on purpose? Shouldn't the frontend drivers, xen/platform-pci.c etc. be configurable without having to enable CONFIG_PARAVIRT? I suppose that would be possible but I don't think it's a useful configuration because you would lose PV spinlocks for example. IIRC the reason this hasn't been implemented until now is that refactoring would be required to the various bits of driver code which assumes PAE + PARAVIRT when they aren't strictly needed, e.g. grant table code. Whether its worth the churn at this stage is debatable, but I think the (in)ability to use PV spinlocks is a red-herring. Adding PV drivers to an HVM guest is a useful thing to do, even without PV spinlocks. PV IO gets you far more incremental benefit than the locks do, adding PV IO paths is the number 1 thing which should be done to any guest. I take this as an ack to change this. :-) It's a best IMHO being able to do this is a useful thing. I can't ack the actual final patch, a) I'm not a relevant maintainer and b) I've not seen the patch. Okay, how do the current config settings regarding Xen look like? ...I'll leave the mechanics down to you and the maintainers to thrash out. Ian. ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] PVHVM drivers in upstream linux kernel
On 12/02/2014 12:36 PM, Ian Campbell wrote: On Tue, 2014-12-02 at 12:33 +0100, Juergen Gross wrote: On 12/02/2014 12:05 PM, Ian Campbell wrote: On Tue, 2014-12-02 at 10:54 +, David Vrabel wrote: On 02/12/14 09:39, Juergen Gross wrote: Hi, looking into the upstream linux sources I realized that the PVHVM drivers of XEN are only available with the pvops kernel. Is this on purpose? Shouldn't the frontend drivers, xen/platform-pci.c etc. be configurable without having to enable CONFIG_PARAVIRT? I suppose that would be possible but I don't think it's a useful configuration because you would lose PV spinlocks for example. IIRC the reason this hasn't been implemented until now is that refactoring would be required to the various bits of driver code which assumes PAE + PARAVIRT when they aren't strictly needed, e.g. grant table code. Whether its worth the churn at this stage is debatable, but I think the (in)ability to use PV spinlocks is a red-herring. Adding PV drivers to an HVM guest is a useful thing to do, even without PV spinlocks. PV IO gets you far more incremental benefit than the locks do, adding PV IO paths is the number 1 thing which should be done to any guest. I take this as an ack to change this. :-) It's a best IMHO being able to do this is a useful thing. I can't ack the actual final patch, a) I'm not a relevant maintainer and b) I've not seen the patch. It was not meant to be taken as an ack for a not yet written patch. :-) Okay, how do the current config settings regarding Xen look like? ...I'll leave the mechanics down to you and the maintainers to thrash out. Yeah, stay tuned. :-) Juergen ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] PVHVM drivers in upstream linux kernel
On 12/02/2014 12:59 PM, David Vrabel wrote: On 02/12/14 11:33, Juergen Gross wrote: I think we have customers wanting to run a default kernel as domU. So it isn't always the distro refusing paravirt, it might be the user... I don't think this is a sensible use case but I'm not adverse to improving the set of Xen config options. To sort things out I'd suggest to: - make XEN independent from PARAVIRT - let XEN_DOM0 select XEN_BACKEND, PARAVIRT, XEN - let XEN_BACKEND select PARAVIRT, XEN (I'd like to be able to build a driver domain without XEN_DOM0) - introduce XEN_FRONTEND, let it select XEN - let frontend drivers and drivers needed by those depend on XEN_FRONTEND - let XEN_PVHVM select XEN_FRONTEND Rather than looking at the current set of configuration options, can you look at what user-visible functionality or use cases need to be covered by a (potentially new) set of configuration options? I'd see: - XEN_PV (selects PARAVIRT, XEN_FRONTEND): be able to run as pv-domain (x86 only) - XEN_PVHVM (selects XEN_FRONTEND): be able to run as hvm-domain with pv-drivers - XEN_BACKEND (selects PARAVIRT if x86): be able to run as driver domain (dom0 or other) - XEN_DOM0 (selects PARAVIRT if x86, XEN_BACKEND): be able to run as dom0 - XEN_FRONTEND: be able to run as domU with pv-drivers Juergen ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] PVHVM drivers in upstream linux kernel
On 02/12/14 13:00, Juergen Gross wrote: I'd see: - XEN_PV (selects PARAVIRT, XEN_FRONTEND): be able to run as pv-domain (x86 only) Depends on PARAVIRT perhaps? - XEN_PVHVM (selects XEN_FRONTEND): be able to run as hvm-domain with pv-drivers - XEN_BACKEND (selects PARAVIRT if x86): be able to run as driver domain (dom0 or other) Does not need to select PARAVIRT -- HVM domains can run backends. - XEN_DOM0 (selects PARAVIRT if x86, XEN_BACKEND): be able to run as dom0 XEN_DOM0 depends on XEN_PV or XEN_PVH (if x86) and whatever ARM needs. - XEN_FRONTEND: be able to run as domU with pv-drivers It may also be interesting to consider splitting the PV MMU stuff under a PARAVIRT_MMU option. This might address a reason why people want to disable PARAVIRT completely. David ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] PVHVM drivers in upstream linux kernel
On Tue, Dec 02, 2014 at 11:05:14AM +, Ian Campbell wrote: On Tue, 2014-12-02 at 10:54 +, David Vrabel wrote: On 02/12/14 09:39, Juergen Gross wrote: Hi, looking into the upstream linux sources I realized that the PVHVM drivers of XEN are only available with the pvops kernel. Is this on purpose? Shouldn't the frontend drivers, xen/platform-pci.c etc. be configurable without having to enable CONFIG_PARAVIRT? I suppose that would be possible but I don't think it's a useful configuration because you would lose PV spinlocks for example. IIRC the reason this hasn't been implemented until now is that refactoring would be required to the various bits of driver code which assumes PAE + PARAVIRT when they aren't strictly needed, e.g. grant table code. Whether its worth the churn at this stage is debatable, but I think the (in)ability to use PV spinlocks is a red-herring. Adding PV drivers to an HVM guest is a useful thing to do, even without PV spinlocks. PV IO gets you far more incremental benefit than the locks do, adding PV IO paths is the number 1 thing which should be done to any guest. One actual usecase is installing from a distro installer which isn't PAE, let alone PARAVIRT enabled[0], to get far enough that you can install a more capable PVHVM kernel with more bells and whistles. If there were distros around who refused wholesale to enable PARAVIRT even in a non-default kernel then it would be more likely that they could be convinced to enable a set of PV IO drivers, since they have 0 impact on a non-PARAVIRT system, and still give significant benefit to Xen users. I don't know of any of the major distros are refusing PARAVIRT in this way though. Ian. [0] The default i386 Debian installer falls into this camp, but you can use the special PV Xen variant to install as PVHVM too so it's not so critical. And the Fedora 21 LiveISO (32-bit) does too. ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] PVHVM drivers in upstream linux kernel
On Tue, Dec 02, 2014 at 03:13:28PM +, Ian Campbell wrote: On Tue, 2014-12-02 at 10:11 -0500, Konrad Rzeszutek Wilk wrote: [0] The default i386 Debian installer falls into this camp, but you can use the special PV Xen variant to install as PVHVM too so it's not so critical. And the Fedora 21 LiveISO (32-bit) does too. Interesting, I thought Fedora had switched to requiring PAE as a minimum baseline everywhere It did, except for the GNOME based LiveCD. And oddly enough it will install an PAE kernel (since the CPU is capable of it) . But of course it won't bundle the Xen drivers in the initramfs, so when the new OS boots it keels over as it unplugs the emulated ones. Ian. ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
