Re: [PATCH v3 00/17] Introducing Linux root partition support for Microsoft Hypervisor
On Tue, Feb 02, 2021 at 10:40:43AM +, David Woodhouse wrote: > On Tue, 2020-12-15 at 16:42 +, Wei Liu wrote: > > On Tue, Dec 15, 2020 at 04:25:03PM +0100, Enrico Weigelt, metux IT consult > > wrote: > > > On 03.12.20 00:22, Wei Liu wrote: > > > > > > Hi, > > > > > > > I don't follow. Do you mean reusing /dev/kvm but with a different set of > > > > APIs underneath? I don't think that will work. > > > > > > My idea was using the same uapi for both hypervisors, so that we can use > > > the same userlands for both. > > > > > > Are the semantis so different that we can't provide the same API ? > > > > We can provide some similar APIs for ease of porting, but can't provide > > 1:1 mappings. By definition KVM and MSHV are two different things. There > > is no goal to make one ABI / API compatible with the other. > > I'm not sure I understand. > > KVM is the Linux userspace API for virtualisation. It is designed to be > versatile enough that it can support multiple implementations across > multiple architectures, including both AMD SVM and Intel VMX on x86. > > Are you saying that KVM has *failed* to be versatile enough that this > can be "just another implementation"? What are the problems? Is it > unfixable? The KVM APIs are good enough to cover guest life cycle management. To make MSHV another implementation of the KVM APIs, we essentially need to massage the data structures both way. They are There is also an aspect for controlling the hypervisor that affect the whole virtualization system. KVM APIs don't handle those. We would need /dev/mshv for that purpose alone. There is another aspect for Microsoft Hypervisor specific features and enhancements, which aren't applicable to KVM. Features make sense for a specific type-1 hypervisor may not make sense for KVM (a type-2 hypervisor). We have no intention to pollute KVM APIs with those. All in all the latter two points make /dev/mshv is a more viable route in the long run. Wei.
Re: [PATCH v3 00/17] Introducing Linux root partition support for Microsoft Hypervisor
On Tue, 2020-12-15 at 16:42 +, Wei Liu wrote: > On Tue, Dec 15, 2020 at 04:25:03PM +0100, Enrico Weigelt, metux IT consult > wrote: > > On 03.12.20 00:22, Wei Liu wrote: > > > > Hi, > > > > > I don't follow. Do you mean reusing /dev/kvm but with a different set of > > > APIs underneath? I don't think that will work. > > > > My idea was using the same uapi for both hypervisors, so that we can use > > the same userlands for both. > > > > Are the semantis so different that we can't provide the same API ? > > We can provide some similar APIs for ease of porting, but can't provide > 1:1 mappings. By definition KVM and MSHV are two different things. There > is no goal to make one ABI / API compatible with the other. I'm not sure I understand. KVM is the Linux userspace API for virtualisation. It is designed to be versatile enough that it can support multiple implementations across multiple architectures, including both AMD SVM and Intel VMX on x86. Are you saying that KVM has *failed* to be versatile enough that this can be "just another implementation"? What are the problems? Is it unfixable? smime.p7s Description: S/MIME cryptographic signature
Re: [PATCH v3 00/17] Introducing Linux root partition support for Microsoft Hypervisor
On Tue, Dec 15, 2020 at 04:25:03PM +0100, Enrico Weigelt, metux IT consult wrote: > On 03.12.20 00:22, Wei Liu wrote: > > Hi, > > > I don't follow. Do you mean reusing /dev/kvm but with a different set of > > APIs underneath? I don't think that will work. > > My idea was using the same uapi for both hypervisors, so that we can use > the same userlands for both. > > Are the semantis so different that we can't provide the same API ? We can provide some similar APIs for ease of porting, but can't provide 1:1 mappings. By definition KVM and MSHV are two different things. There is no goal to make one ABI / API compatible with the other. > > > In any case, the first version of /dev/mshv was posted a few days ago > > [0]. While we've chosen to follow closely KVM's model, Microsoft > > Hypervisor has its own APIs. > > I have to admit, I don't know much about hyperv - what are the main > differences (from userland perspective) between hyperv and kvm ? > They have different architecture and hence different ways to deal with things. The difference will inevitably make its way to userland. Without going into all the details, you can have a look how Xen and KVM differ architecturally. That will give you a pretty good idea on the differences. Wei. > > --mtx > > -- > --- > Hinweis: unverschlüsselte E-Mails können leicht abgehört und manipuliert > werden ! Für eine vertrauliche Kommunikation senden Sie bitte ihren > GPG/PGP-Schlüssel zu. > --- > Enrico Weigelt, metux IT consult > Free software and Linux embedded engineering > i...@metux.net -- +49-151-27565287
Re: [PATCH v3 00/17] Introducing Linux root partition support for Microsoft Hypervisor
On 03.12.20 00:22, Wei Liu wrote: Hi, > I don't follow. Do you mean reusing /dev/kvm but with a different set of > APIs underneath? I don't think that will work. My idea was using the same uapi for both hypervisors, so that we can use the same userlands for both. Are the semantis so different that we can't provide the same API ? > In any case, the first version of /dev/mshv was posted a few days ago > [0]. While we've chosen to follow closely KVM's model, Microsoft > Hypervisor has its own APIs. I have to admit, I don't know much about hyperv - what are the main differences (from userland perspective) between hyperv and kvm ? --mtx -- --- Hinweis: unverschlüsselte E-Mails können leicht abgehört und manipuliert werden ! Für eine vertrauliche Kommunikation senden Sie bitte ihren GPG/PGP-Schlüssel zu. --- Enrico Weigelt, metux IT consult Free software and Linux embedded engineering i...@metux.net -- +49-151-27565287
Re: [PATCH v3 00/17] Introducing Linux root partition support for Microsoft Hypervisor
On Wed, Dec 02, 2020 at 08:51:38PM +0100, Enrico Weigelt, metux IT consult wrote: > On 24.11.20 18:07, Wei Liu wrote: > > Hi, > > > There will be a subsequent patch series to provide a > > device node (/dev/mshv) such that userspace programs can create and run > > virtual > > machines. > > Any chance of using the already existing /dev/kvm interface ? > I don't follow. Do you mean reusing /dev/kvm but with a different set of APIs underneath? I don't think that will work. In any case, the first version of /dev/mshv was posted a few days ago [0]. While we've chosen to follow closely KVM's model, Microsoft Hypervisor has its own APIs. Wei. 0: https://lore.kernel.org/linux-hyperv/1605918637-12192-1-git-send-email-nunodasne...@linux.microsoft.com/ > --mtx > > -- > --- > Hinweis: unverschlüsselte E-Mails können leicht abgehört und manipuliert > werden ! Für eine vertrauliche Kommunikation senden Sie bitte ihren > GPG/PGP-Schlüssel zu. > --- > Enrico Weigelt, metux IT consult > Free software and Linux embedded engineering > i...@metux.net -- +49-151-27565287
Re: [PATCH v3 00/17] Introducing Linux root partition support for Microsoft Hypervisor
On 24.11.20 18:07, Wei Liu wrote: Hi, > There will be a subsequent patch series to provide a > device node (/dev/mshv) such that userspace programs can create and run > virtual > machines. Any chance of using the already existing /dev/kvm interface ? --mtx -- --- Hinweis: unverschlüsselte E-Mails können leicht abgehört und manipuliert werden ! Für eine vertrauliche Kommunikation senden Sie bitte ihren GPG/PGP-Schlüssel zu. --- Enrico Weigelt, metux IT consult Free software and Linux embedded engineering i...@metux.net -- +49-151-27565287
[PATCH v3 00/17] Introducing Linux root partition support for Microsoft Hypervisor
Hi all Here we propose this patch series to make Linux run as the root partition [0] on Microsoft Hypervisor [1]. There will be a subsequent patch series to provide a device node (/dev/mshv) such that userspace programs can create and run virtual machines. We've also ported Cloud Hypervisor [3] over and have been able to boot a Linux guest with Virtio devices since late July. Being an RFC sereis, this implements only the absolutely necessary components to get things running. I will break down this series a bit. A large portion of this series consists of patches that augment hyperv-tlfs.h. They should be rather uncontroversial and can be applied right away. A few key things other than the changes to hyperv-tlfs.h: 1. Linux needs to setup existing Hyper-V facilities differently. 2. Linux needs to make a few hypercalls to bring up APs. 3. Interrupts are remapped by IOMMU, which is controlled by the hypervisor. Linux needs to make hypercalls to map and unmap interrupts. This is done by introducing a new MSI irqdomain and new irqchips. This series is now based on 5.10-rc1. And thanks to tglx's overhaul of the MSI code, our implementation of the MSI irq domain is shorter. Comments and suggestions are welcome. Thanks, Wei. Cc: sa...@linux.intel.com Cc: robert.bradf...@intel.com Cc: sebastien.bo...@intel.com Changes since v2: 1. Address more comments from Vitaly. 2. Fix and test 32bit build. Changes since v1: 1. Simplify MSI IRQ domain implementation. 2. Address Vitaly's comments. Wei Liu (17): asm-generic/hyperv: change HV_CPU_POWER_MANAGEMENT to HV_CPU_MANAGEMENT x86/hyperv: detect if Linux is the root partition Drivers: hv: vmbus: skip VMBus initialization if Linux is root iommu/hyperv: don't setup IRQ remapping when running as root clocksource/hyperv: use MSR-based access if running as root x86/hyperv: allocate output arg pages if required x86/hyperv: extract partition ID from Microsoft Hypervisor if necessary x86/hyperv: handling hypercall page setup for root x86/hyperv: provide a bunch of helper functions x86/hyperv: implement and use hv_smp_prepare_cpus asm-generic/hyperv: update hv_msi_entry asm-generic/hyperv: update hv_interrupt_entry asm-generic/hyperv: introduce hv_device_id and auxiliary structures asm-generic/hyperv: import data structures for mapping device interrupts x86/hyperv: implement an MSI domain for root partition x86/ioapic: export a few functions and data structures via io_apic.h x86/hyperv: handle IO-APIC when running as root arch/x86/hyperv/Makefile| 4 +- arch/x86/hyperv/hv_init.c | 121 +- arch/x86/hyperv/hv_proc.c | 215 +++ arch/x86/hyperv/irqdomain.c | 556 arch/x86/include/asm/hyperv-tlfs.h | 23 ++ arch/x86/include/asm/io_apic.h | 21 ++ arch/x86/include/asm/mshyperv.h | 13 +- arch/x86/kernel/apic/io_apic.c | 28 +- arch/x86/kernel/cpu/mshyperv.c | 49 +++ drivers/clocksource/hyperv_timer.c | 3 + drivers/hv/vmbus_drv.c | 3 + drivers/iommu/hyperv-iommu.c| 3 +- drivers/pci/controller/pci-hyperv.c | 2 +- include/asm-generic/hyperv-tlfs.h | 254 - 14 files changed, 1257 insertions(+), 38 deletions(-) create mode 100644 arch/x86/hyperv/hv_proc.c create mode 100644 arch/x86/hyperv/irqdomain.c -- 2.20.1
