Re: [Qemu-devel] [kvmarm] [RFC] Virtio-desktop: Virtio-based virtual desktop
On 27.01.2013, at 15:07, Anthony Liguori wrote: > Anup Patel writes: > >> Hi All, >> >> How about having a generic Virtio-based machine for emulating a virtual >> desktop ? >> >> I know folks have already thought about this and probably also tried >> something or other on this front but, it will be good to know the downsides. >> >> Virtio-desktop can be a separate specification describing a virtual >> desktop. >> Of-course we cannot avoid few architecture dependent virtual devices in but >> the Virtio-desktop specification will try to keep minimum possible >> architecture dependent devices. > > There's a lot of reasons why a pure PV machine type is a bad idea. Lots > of people have enumerated them in this thread. > > But let me mention some things that I think we don't have covered today > with PV: > > - Graphics. Yes, I know QXL exists but it's (a) dependent on PCI (b) > lacks the ability to gracefully degrade making it hopelessly tied to > spice. There was a QXL-on-virtio port in the works a while ago IIRC: > - Input. PS/2 mouse provides relative input which sucks in guests. > For absolute input, we have VMMouse which is x86-specific, USB > tablets (which are expensive to emulate) or the spice mouse which is > intimately tied to the full Spice stack. I thought the USB tablet is ok today thanks to auto-suspend of the bus? Or was that only with ehci? > > - Guest interaction. Copy/paste, drag and drop, etc. In theory this > is covered in spice agents but it's all again hopelessly tied to > Spice which makes it non-portable. - Keyboard. When running with VNC, the 3 stacks involved in converting keyboard layouts back and forth are really confusing to users. > So there's good work todo but it's almost certainly in working with the > Spice community to try to make what they already have more accessible to > non-x86 architectures. Hooray :) Alex
Re: [Qemu-devel] [kvmarm] [RFC] Virtio-desktop: Virtio-based virtual desktop
On Sun, Jan 27, 2013 at 10:12:20AM +, Blue Swirl wrote: > On Sun, Jan 27, 2013 at 8:20 AM, Gleb Natapov wrote: > > On Fri, Jan 25, 2013 at 08:31:54PM +0100, Alexander Graf wrote: > >> > >> On 25.01.2013, at 20:04, Blue Swirl wrote: > >> > >> > On Thu, Jan 24, 2013 at 6:10 AM, Anup Patel > >> > wrote: > >> >> Hi All, > >> >> > >> >> How about having a generic Virtio-based machine for emulating a virtual > >> >> desktop ? > >> > > >> > I have also thought about this, current virtio design is not very > >> > clean. On the downside, pure no-legacy approach might not work well if > >> > you want the host to give control of a host device to guest (VFIO like > >> > pass through using IOMMUs). > >> > > >> >> I know folks have already thought about this and probably also tried > >> >> something or other on this front but, it will be good to know the > >> >> downsides. > >> >> > >> >> Virtio-desktop can be a separate specification describing a virtual > >> >> desktop. > >> >> Of-course we cannot avoid few architecture dependent virtual devices in > >> >> but > >> >> the Virtio-desktop specification will try to keep minimum possible > >> >> architecture dependent devices. > >> >> > >> >> As per our thoughts, a Virtio-desktop will have two kinds of devices: > >> >> 1. Architecture dependent devices: This devices will be emulated > >> >> in-kernel > >> >> by architecture specific code of KVM or Xen or Some other hypervisor. > >> >> a) Virtualized CPU > >> >> b) Virtualized PIC (i.e. in-kernel architecture specific emulation of > >> >> irqchip) > >> >> c) Virtualized Timer (i.e. in-kernel architecture specific emulation > >> >> of > >> >> timer chip) > >> > > >> > I think reusing PIC and timer design is not the most optimal. What a > >> > PV aware OS really wants is to wake up because of an external event or > >> > at some specific point of time (or after a specific delay) and easy > >> > way to manage the VM clock without timer tick counting. With a > >> > tickless approach, it would need the timer events as rarely as > >> > possible. > >> > > >> > Perhaps a better approach would be to introduce a way for the guest to > >> > subscribe to a list of external events (including time), which would > >> > be fed to it via something like reverse hypercall (or just use legacy > >> > interrupts). Preferably it should be possible to pass any events > >> > through a stack of guests to the end consumer guest and even > >> > applications in a guest. > >> > >> This approach falls apart once hardware vendors implement timer interrupts > >> inside a guest context (which Intel and IIUC ARM are doing). At that > >> point, it's actually more efficient to do real timer calls to real > >> hardware. > >> > > Same with irq chip. Implementing PV irqchip today is going backwards. > > I'm not sure this should be the conclusion. Obviously throwing more > hardware at virtualization makes it faster and the x86 host will > probably also benefit from faster interrupt handling as a side > benefit, so emulating the PIC etc. makes sense for now and also in > short term (some years from now). Moder OSes, i.e those that have a non zero chance to get virtio-desktop support, do not use PIC on x86 after bootup, so there is not performance gains in replacing it with something PV and I hope you do not propose that. You probably think about using PV in place of LAPIC which, besides have to be written and debugged, will be slower on modern x86 HW since mode x86 HW accelerates LAPIC emulation. AFAIK arm also has irq chip virtualization support in HW. > > But long term, if you could direct the hardware design efforts of > major companies and standardization bodies to any direction, is fast > emulation of 30 year old technology what would create the fastest > architecture in the future? For example, maybe the PIC+LAPIC should be > replaced in long term by something that can vector the interrupts to > different VCPUs directly, each at different privilege levels? > What prevents you from doing it today with LAPIC/MSIX? PIC/IOAPIC are legacy and not used for anything performance sensitive. And do you truly believe that HW vendors will abandon current HW in favor of your design? -- Gleb.
Re: [Qemu-devel] [kvmarm] [RFC] Virtio-desktop: Virtio-based virtual desktop
On Sun, Jan 27, 2013 at 8:20 AM, Gleb Natapov wrote: > On Fri, Jan 25, 2013 at 08:31:54PM +0100, Alexander Graf wrote: >> >> On 25.01.2013, at 20:04, Blue Swirl wrote: >> >> > On Thu, Jan 24, 2013 at 6:10 AM, Anup Patel wrote: >> >> Hi All, >> >> >> >> How about having a generic Virtio-based machine for emulating a virtual >> >> desktop ? >> > >> > I have also thought about this, current virtio design is not very >> > clean. On the downside, pure no-legacy approach might not work well if >> > you want the host to give control of a host device to guest (VFIO like >> > pass through using IOMMUs). >> > >> >> I know folks have already thought about this and probably also tried >> >> something or other on this front but, it will be good to know the >> >> downsides. >> >> >> >> Virtio-desktop can be a separate specification describing a virtual >> >> desktop. >> >> Of-course we cannot avoid few architecture dependent virtual devices in >> >> but >> >> the Virtio-desktop specification will try to keep minimum possible >> >> architecture dependent devices. >> >> >> >> As per our thoughts, a Virtio-desktop will have two kinds of devices: >> >> 1. Architecture dependent devices: This devices will be emulated in-kernel >> >> by architecture specific code of KVM or Xen or Some other hypervisor. >> >> a) Virtualized CPU >> >> b) Virtualized PIC (i.e. in-kernel architecture specific emulation of >> >> irqchip) >> >> c) Virtualized Timer (i.e. in-kernel architecture specific emulation of >> >> timer chip) >> > >> > I think reusing PIC and timer design is not the most optimal. What a >> > PV aware OS really wants is to wake up because of an external event or >> > at some specific point of time (or after a specific delay) and easy >> > way to manage the VM clock without timer tick counting. With a >> > tickless approach, it would need the timer events as rarely as >> > possible. >> > >> > Perhaps a better approach would be to introduce a way for the guest to >> > subscribe to a list of external events (including time), which would >> > be fed to it via something like reverse hypercall (or just use legacy >> > interrupts). Preferably it should be possible to pass any events >> > through a stack of guests to the end consumer guest and even >> > applications in a guest. >> >> This approach falls apart once hardware vendors implement timer interrupts >> inside a guest context (which Intel and IIUC ARM are doing). At that point, >> it's actually more efficient to do real timer calls to real hardware. >> > Same with irq chip. Implementing PV irqchip today is going backwards. I'm not sure this should be the conclusion. Obviously throwing more hardware at virtualization makes it faster and the x86 host will probably also benefit from faster interrupt handling as a side benefit, so emulating the PIC etc. makes sense for now and also in short term (some years from now). But long term, if you could direct the hardware design efforts of major companies and standardization bodies to any direction, is fast emulation of 30 year old technology what would create the fastest architecture in the future? For example, maybe the PIC+LAPIC should be replaced in long term by something that can vector the interrupts to different VCPUs directly, each at different privilege levels? > >> PV is bad. We only do it when we have to. Not doing PV where we don't have >> to is exactly the reason KVM is superior to Xen. >> > Exactly. Implementing PV for something that does not require PV (for > performance reasons mostly) is trading tested guest code, to untested, > and unwritten, one. HW acceleration of Xen would just need faster IPC, but that has probably reached max performance ages ago. The legacy driver benefit is a good point, though more so for short term. > > -- > Gleb.
Re: [Qemu-devel] [kvmarm] [RFC] Virtio-desktop: Virtio-based virtual desktop
On Fri, Jan 25, 2013 at 08:31:54PM +0100, Alexander Graf wrote: > > On 25.01.2013, at 20:04, Blue Swirl wrote: > > > On Thu, Jan 24, 2013 at 6:10 AM, Anup Patel wrote: > >> Hi All, > >> > >> How about having a generic Virtio-based machine for emulating a virtual > >> desktop ? > > > > I have also thought about this, current virtio design is not very > > clean. On the downside, pure no-legacy approach might not work well if > > you want the host to give control of a host device to guest (VFIO like > > pass through using IOMMUs). > > > >> I know folks have already thought about this and probably also tried > >> something or other on this front but, it will be good to know the > >> downsides. > >> > >> Virtio-desktop can be a separate specification describing a virtual > >> desktop. > >> Of-course we cannot avoid few architecture dependent virtual devices in but > >> the Virtio-desktop specification will try to keep minimum possible > >> architecture dependent devices. > >> > >> As per our thoughts, a Virtio-desktop will have two kinds of devices: > >> 1. Architecture dependent devices: This devices will be emulated in-kernel > >> by architecture specific code of KVM or Xen or Some other hypervisor. > >> a) Virtualized CPU > >> b) Virtualized PIC (i.e. in-kernel architecture specific emulation of > >> irqchip) > >> c) Virtualized Timer (i.e. in-kernel architecture specific emulation of > >> timer chip) > > > > I think reusing PIC and timer design is not the most optimal. What a > > PV aware OS really wants is to wake up because of an external event or > > at some specific point of time (or after a specific delay) and easy > > way to manage the VM clock without timer tick counting. With a > > tickless approach, it would need the timer events as rarely as > > possible. > > > > Perhaps a better approach would be to introduce a way for the guest to > > subscribe to a list of external events (including time), which would > > be fed to it via something like reverse hypercall (or just use legacy > > interrupts). Preferably it should be possible to pass any events > > through a stack of guests to the end consumer guest and even > > applications in a guest. > > This approach falls apart once hardware vendors implement timer interrupts > inside a guest context (which Intel and IIUC ARM are doing). At that point, > it's actually more efficient to do real timer calls to real hardware. > Same with irq chip. Implementing PV irqchip today is going backwards. > PV is bad. We only do it when we have to. Not doing PV where we don't have to > is exactly the reason KVM is superior to Xen. > Exactly. Implementing PV for something that does not require PV (for performance reasons mostly) is trading tested guest code, to untested, and unwritten, one. -- Gleb.
Re: [Qemu-devel] [kvmarm] [RFC] Virtio-desktop: Virtio-based virtual desktop
On 25.01.2013, at 20:04, Blue Swirl wrote: > On Thu, Jan 24, 2013 at 6:10 AM, Anup Patel wrote: >> Hi All, >> >> How about having a generic Virtio-based machine for emulating a virtual >> desktop ? > > I have also thought about this, current virtio design is not very > clean. On the downside, pure no-legacy approach might not work well if > you want the host to give control of a host device to guest (VFIO like > pass through using IOMMUs). > >> I know folks have already thought about this and probably also tried >> something or other on this front but, it will be good to know the downsides. >> >> Virtio-desktop can be a separate specification describing a virtual desktop. >> Of-course we cannot avoid few architecture dependent virtual devices in but >> the Virtio-desktop specification will try to keep minimum possible >> architecture dependent devices. >> >> As per our thoughts, a Virtio-desktop will have two kinds of devices: >> 1. Architecture dependent devices: This devices will be emulated in-kernel >> by architecture specific code of KVM or Xen or Some other hypervisor. >> a) Virtualized CPU >> b) Virtualized PIC (i.e. in-kernel architecture specific emulation of >> irqchip) >> c) Virtualized Timer (i.e. in-kernel architecture specific emulation of >> timer chip) > > I think reusing PIC and timer design is not the most optimal. What a > PV aware OS really wants is to wake up because of an external event or > at some specific point of time (or after a specific delay) and easy > way to manage the VM clock without timer tick counting. With a > tickless approach, it would need the timer events as rarely as > possible. > > Perhaps a better approach would be to introduce a way for the guest to > subscribe to a list of external events (including time), which would > be fed to it via something like reverse hypercall (or just use legacy > interrupts). Preferably it should be possible to pass any events > through a stack of guests to the end consumer guest and even > applications in a guest. This approach falls apart once hardware vendors implement timer interrupts inside a guest context (which Intel and IIUC ARM are doing). At that point, it's actually more efficient to do real timer calls to real hardware. PV is bad. We only do it when we have to. Not doing PV where we don't have to is exactly the reason KVM is superior to Xen. Alex
Re: [Qemu-devel] [kvmarm] [RFC] Virtio-desktop: Virtio-based virtual desktop
On Thu, Jan 24, 2013 at 8:08 PM, Alexander Graf wrote: > > On 24.01.2013, at 10:25, Stefan Hajnoczi wrote: > > > On Thu, Jan 24, 2013 at 11:40:24AM +0530, Anup Patel wrote: > >> IMHO, If we have something like Virtio-desktop specification then all > >> possible guest OSes can have support for it and different hypervisor can > >> emulate it without worrying about guest support. > > > > At this point x86 virtualization is mature and working with a mix of > > emulated x86 architecture pieces and virtio devices for > > performance-critical or open-ended functionality that we want to be able > > to extend. > > > > ARM is getting KVM and virtio-mmio support. It will be in a similar > > position soon. > > > > Virtio guest drivers have not been implemented widely. The Linux and > > Windows efforts are driven by the folks who were behind virtio from the > > start, but Solaris, FreeBSD, and others didn't really jump on the virtio > > bandwagon. > > > > Given this landscape, what is the advantage of doing a virtio-desktop? > > It will still need to fall back on ARM or x86 which is already being > > virtualized and emulated. > > > > Depending on how you see it we either have virtio-desktop already or, > > if not, I think the experience with virtio adoption suggests other > > hypervisors and guest OSes will not trip over themselves to implement > > virtio-desktop. > > > > What's the advantage over virtualizating an existing ARM or x86 platform > > and using virtio devices where appropriate? > > You don't get changing hardware for changing CPUs. I don't think it makes > sense to do a cross-arch virtio-desktop machine type. Different > architectures simply have different needs. > [Anup] Virtio-desktop does not rule-out archictecture needs. In fact, Virtio-desktop specfication can have architecture specific devices and architecture specific requirements. The most important point here is to have VM specification which mostly prefers Virtio devices. > > But check out the QEMU e500 machine. We have a fully device tree based > machine type in the kernel. QEMU drives it by generating a device tree for > devices it actually exposes on the fly. > > The big advantage we have here is that > > 1) We don't have to emulate all hardware real hardware emulates > 2) We're not restricted to emulate what real hardware emulates. PCI on > ARM anyone? > 3) Different CPU types can live on the same machine. This is something > that x86 is doing already. When you get a SoC, guests are usually > guaranteed a core <-> device correlation though. > > So overall, having a PV machine makes sense. Having the same common PV > machine standardized across different architectures does not make sense. > [Anup] Agreed. Virtio-desktop = Architecture dependent devices + Architecture requirements + Virtio devices. [Anup] Virtio-desktop specification will remain incomplete without incorporating architecture requirements. [Anup] I think it is possible to have a common Virtio-desktop specification which stresses on maximum use of Virtio devices and also includes architecture specific virtualization needs. > > > Alex > > > ___ > kvmarm mailing list > kvm...@lists.cs.columbia.edu > https://lists.cs.columbia.edu/cucslists/listinfo/kvmarm > --Anup
Re: [Qemu-devel] [kvmarm] [RFC] Virtio-desktop: Virtio-based virtual desktop
On 24.01.2013, at 15:42, Peter Maydell wrote: > On 24 January 2013 14:38, Alexander Graf wrote: >> But check out the QEMU e500 machine. We have a fully device tree >> based machine type in the kernel. QEMU drives it by generating a >> device tree for devices it actually exposes on the fly. > > The ARM equivalent for that would be mach-virt, I think > (I forget what the status is but no doubt somebody will > remind me :-)) > >> The big advantage we have here is that >> >> 1) We don't have to emulate all hardware real hardware emulates >> 2) We're not restricted to emulate what real hardware emulates. PCI on ARM >> anyone? > > ...do we have an ARM PCI controller of any working kind in > the kernel? versatilepb's PCI controller doesn't count as > it is utterly broken :-) Don't the Marvell chips have PCI? And Tegra? I'm sure there have to be some SoCs with PCI :). Alex
Re: [Qemu-devel] [kvmarm] [RFC] Virtio-desktop: Virtio-based virtual desktop
On 24 January 2013 14:38, Alexander Graf wrote: > But check out the QEMU e500 machine. We have a fully device tree > based machine type in the kernel. QEMU drives it by generating a > device tree for devices it actually exposes on the fly. The ARM equivalent for that would be mach-virt, I think (I forget what the status is but no doubt somebody will remind me :-)) > The big advantage we have here is that > > 1) We don't have to emulate all hardware real hardware emulates > 2) We're not restricted to emulate what real hardware emulates. PCI on ARM > anyone? ...do we have an ARM PCI controller of any working kind in the kernel? versatilepb's PCI controller doesn't count as it is utterly broken :-) -- PMM
Re: [Qemu-devel] [kvmarm] [RFC] Virtio-desktop: Virtio-based virtual desktop
On 24.01.2013, at 10:25, Stefan Hajnoczi wrote: > On Thu, Jan 24, 2013 at 11:40:24AM +0530, Anup Patel wrote: >> IMHO, If we have something like Virtio-desktop specification then all >> possible guest OSes can have support for it and different hypervisor can >> emulate it without worrying about guest support. > > At this point x86 virtualization is mature and working with a mix of > emulated x86 architecture pieces and virtio devices for > performance-critical or open-ended functionality that we want to be able > to extend. > > ARM is getting KVM and virtio-mmio support. It will be in a similar > position soon. > > Virtio guest drivers have not been implemented widely. The Linux and > Windows efforts are driven by the folks who were behind virtio from the > start, but Solaris, FreeBSD, and others didn't really jump on the virtio > bandwagon. > > Given this landscape, what is the advantage of doing a virtio-desktop? > It will still need to fall back on ARM or x86 which is already being > virtualized and emulated. > > Depending on how you see it we either have virtio-desktop already or, > if not, I think the experience with virtio adoption suggests other > hypervisors and guest OSes will not trip over themselves to implement > virtio-desktop. > > What's the advantage over virtualizating an existing ARM or x86 platform > and using virtio devices where appropriate? You don't get changing hardware for changing CPUs. I don't think it makes sense to do a cross-arch virtio-desktop machine type. Different architectures simply have different needs. But check out the QEMU e500 machine. We have a fully device tree based machine type in the kernel. QEMU drives it by generating a device tree for devices it actually exposes on the fly. The big advantage we have here is that 1) We don't have to emulate all hardware real hardware emulates 2) We're not restricted to emulate what real hardware emulates. PCI on ARM anyone? 3) Different CPU types can live on the same machine. This is something that x86 is doing already. When you get a SoC, guests are usually guaranteed a core <-> device correlation though. So overall, having a PV machine makes sense. Having the same common PV machine standardized across different architectures does not make sense. Alex
