RE: [PATCH v3 0/8] KVM-VFIO IRQ forward control
> -Original Message- > From: Eric Auger [mailto:eric.au...@linaro.org] > Sent: Monday, November 24, 2014 4:27 PM > To: Wu, Feng; eric.au...@st.com; christoffer.d...@linaro.org; > marc.zyng...@arm.com; linux-arm-ker...@lists.infradead.org; > kvm...@lists.cs.columbia.edu; k...@vger.kernel.org; > alex.william...@redhat.com; joel.sch...@amd.com; > kim.phill...@freescale.com; pau...@samba.org; g...@kernel.org; > pbonz...@redhat.com; ag...@suse.de > Cc: linux-kernel@vger.kernel.org; patc...@linaro.org; will.dea...@arm.com; > a.mota...@virtualopensystems.com; a.r...@virtualopensystems.com; > john.li...@huawei.com; ming@canonical.com > Subject: Re: [PATCH v3 0/8] KVM-VFIO IRQ forward control > > On 11/24/2014 09:14 AM, Wu, Feng wrote: > > > > > >> -Original Message- > >> From: Eric Auger [mailto:eric.au...@linaro.org] > >> Sent: Monday, November 24, 2014 2:36 AM > >> To: eric.au...@st.com; eric.au...@linaro.org; christoffer.d...@linaro.org; > >> marc.zyng...@arm.com; linux-arm-ker...@lists.infradead.org; > >> kvm...@lists.cs.columbia.edu; k...@vger.kernel.org; > >> alex.william...@redhat.com; joel.sch...@amd.com; > >> kim.phill...@freescale.com; pau...@samba.org; g...@kernel.org; > >> pbonz...@redhat.com; ag...@suse.de > >> Cc: linux-kernel@vger.kernel.org; patc...@linaro.org; > will.dea...@arm.com; > >> a.mota...@virtualopensystems.com; a.r...@virtualopensystems.com; > >> john.li...@huawei.com; ming@canonical.com; Wu, Feng > >> Subject: [PATCH v3 0/8] KVM-VFIO IRQ forward control > >> > >> This series proposes an integration of "ARM: Forwarding physical > >> interrupts to a guest VM" (http://lwn.net/Articles/603514/) in > >> KVM. > >> > >> It enables to transform a VFIO platform driver IRQ into a forwarded > >> IRQ. > >> > >> When a physical IRQ is forwarded (to a guest), the host does not > >> deactivates this latter. Completion ownership is transferred to the > >> guest. When the guest deactivates the associated virtual IRQ, > >> the interrupt controler automatically completes the physical IRQ. > >> Obviously this requires some dedicated HW support in the interrupt > >> controler. > >> > >> The direct benefit is that, for a level sensitive IRQ, it avoids a > >> VM exit on forwarded IRQ completion. > >> > >> When the IRQ is forwarded, the VFIO platform driver does not need to > >> mask the physical IRQ anymore before signaling the eventfd. Indeed > >> genirq lowers the running priority, enabling other physical IRQ to hit > >> except that one. > >> > >> Besides, the injection still is based on irqfd triggering. The only > >> impact on irqfd process is resamplefd is not called anymore on > >> virtual IRQ completion since this latter becomes "transparent". > >> > >> The current integration is based on an extension of the KVM-VFIO > >> device, previously used by KVM to interact with VFIO groups. The > >> patch series now enables KVM to directly interact with a VFIO > >> platform device. The VFIO external API was extended for that purpose. > >> > >> Th KVM-VFIO device can get/put the vfio platform device, check its > >> integrity and type, get the IRQ number associated to an IRQ index. > >> > >> The IRQ forward programming is architecture specific (virtual interrupt > >> controller programming basically). However the whole infrastructure is > >> kept generic. > >> > >> from a user point of view, the functionality is provided through a > >> new KVM-VFIO group named KVM_DEV_VFIO_DEVICE and 2 associated > >> attributes: > >> - KVM_DEV_VFIO_DEVICE_FORWARD_IRQ, > >> - KVM_DEV_VFIO_DEVICE_UNFORWARD_IRQ. > >> > >> The capability can be checked with KVM_HAS_DEVICE_ATTR. > >> > >> Forwarding must be activated before VFIO signaling mechanism is set > >> using VFIO_DEVICE_SET_IRQS and unset while the signaling is disabled. > >> > >> --- > >> > >> This patch series has the following dependencies: > >> - "ARM: Forwarding physical interrupts to a guest VM" > >> (http://lwn.net/Articles/603514/) > >> - [PATCH v9 00/19] VFIO support for platform and AMBA devices on ARM > >> (http://www.spinics.net/lists/kvm-arm/msg11745.html) > >> - [PATCH v2 0/6] vfio: type1: support for ARM SMMUS with > >> VFIO_IOMMU_TYPE1 > >> (http://www.spinics.net/lists/kvm-arm/msg11738.ht
Re: [PATCH v3 0/8] KVM-VFIO IRQ forward control
On 11/24/2014 09:14 AM, Wu, Feng wrote: > > >> -Original Message- >> From: Eric Auger [mailto:eric.au...@linaro.org] >> Sent: Monday, November 24, 2014 2:36 AM >> To: eric.au...@st.com; eric.au...@linaro.org; christoffer.d...@linaro.org; >> marc.zyng...@arm.com; linux-arm-ker...@lists.infradead.org; >> kvm...@lists.cs.columbia.edu; k...@vger.kernel.org; >> alex.william...@redhat.com; joel.sch...@amd.com; >> kim.phill...@freescale.com; pau...@samba.org; g...@kernel.org; >> pbonz...@redhat.com; ag...@suse.de >> Cc: linux-kernel@vger.kernel.org; patc...@linaro.org; will.dea...@arm.com; >> a.mota...@virtualopensystems.com; a.r...@virtualopensystems.com; >> john.li...@huawei.com; ming@canonical.com; Wu, Feng >> Subject: [PATCH v3 0/8] KVM-VFIO IRQ forward control >> >> This series proposes an integration of "ARM: Forwarding physical >> interrupts to a guest VM" (http://lwn.net/Articles/603514/) in >> KVM. >> >> It enables to transform a VFIO platform driver IRQ into a forwarded >> IRQ. >> >> When a physical IRQ is forwarded (to a guest), the host does not >> deactivates this latter. Completion ownership is transferred to the >> guest. When the guest deactivates the associated virtual IRQ, >> the interrupt controler automatically completes the physical IRQ. >> Obviously this requires some dedicated HW support in the interrupt >> controler. >> >> The direct benefit is that, for a level sensitive IRQ, it avoids a >> VM exit on forwarded IRQ completion. >> >> When the IRQ is forwarded, the VFIO platform driver does not need to >> mask the physical IRQ anymore before signaling the eventfd. Indeed >> genirq lowers the running priority, enabling other physical IRQ to hit >> except that one. >> >> Besides, the injection still is based on irqfd triggering. The only >> impact on irqfd process is resamplefd is not called anymore on >> virtual IRQ completion since this latter becomes "transparent". >> >> The current integration is based on an extension of the KVM-VFIO >> device, previously used by KVM to interact with VFIO groups. The >> patch series now enables KVM to directly interact with a VFIO >> platform device. The VFIO external API was extended for that purpose. >> >> Th KVM-VFIO device can get/put the vfio platform device, check its >> integrity and type, get the IRQ number associated to an IRQ index. >> >> The IRQ forward programming is architecture specific (virtual interrupt >> controller programming basically). However the whole infrastructure is >> kept generic. >> >> from a user point of view, the functionality is provided through a >> new KVM-VFIO group named KVM_DEV_VFIO_DEVICE and 2 associated >> attributes: >> - KVM_DEV_VFIO_DEVICE_FORWARD_IRQ, >> - KVM_DEV_VFIO_DEVICE_UNFORWARD_IRQ. >> >> The capability can be checked with KVM_HAS_DEVICE_ATTR. >> >> Forwarding must be activated before VFIO signaling mechanism is set >> using VFIO_DEVICE_SET_IRQS and unset while the signaling is disabled. >> >> --- >> >> This patch series has the following dependencies: >> - "ARM: Forwarding physical interrupts to a guest VM" >> (http://lwn.net/Articles/603514/) >> - [PATCH v9 00/19] VFIO support for platform and AMBA devices on ARM >> (http://www.spinics.net/lists/kvm-arm/msg11745.html) >> - [PATCH v2 0/6] vfio: type1: support for ARM SMMUS with >> VFIO_IOMMU_TYPE1 >> (http://www.spinics.net/lists/kvm-arm/msg11738.html) >> >> Integrated pieces can be found at >> ssh://git.linaro.org/people/eric.auger/linux.git >> on branch irqfd_integ_v8 >> >> This was was tested on Calxeda Midway, assigning the xgmac main IRQ. >> > > Hi Eric, > > Did you send out the latest QEMU part for this patch set, I notice that v6 of > The QEMU part is sent out, but seems some structure in this new version > has been changed, such as, struct kvm_arch_forwarded_irq (subindex is added > in this version), so a new patchset in QEMU is also needed. Hi Feng, v7 is available at: http://lists.gnu.org/archive/html/qemu-devel/2014-10/msg03804.html. It already illustrates KVM-VFIO device usage. v8 which will indeed integrate subindex addition will be delivered this week. Best Regards Eric > > Thanks, > Feng > > >> v2 -> v3: >> - kvm_fwd_irq_action enum replaced by a bool (KVM_VFIO_IRQ_CLEANUP does >> not >> exist anymore) >> - a new struct local to vfio.c was introduced to wrap kvm_fw_irq and make it >> linkable: kvm_vfio_fwd_irq_node >> - kvm_fwd_irq now is self-contained (includes struct vfio_device *) >> - a single list of kvm_vfio_fwd_irq_irq_node is used instead of having >> a list of devices and a list of forward irq per device. Having 2 lists >> brought extra complexity. >> - the VFIO device ref counter is incremented each time a new IRQ is >> forwarded. >> It is not attempted anymore to hold a single reference whatever the number >> of forwarded IRQs. >> - subindex added on top of index to be closer to VFIO API >> - platform device check moved in the arm specific implementation >> - enable the KVM-VFIO
RE: [PATCH v3 0/8] KVM-VFIO IRQ forward control
> -Original Message- > From: Eric Auger [mailto:eric.au...@linaro.org] > Sent: Monday, November 24, 2014 2:36 AM > To: eric.au...@st.com; eric.au...@linaro.org; christoffer.d...@linaro.org; > marc.zyng...@arm.com; linux-arm-ker...@lists.infradead.org; > kvm...@lists.cs.columbia.edu; k...@vger.kernel.org; > alex.william...@redhat.com; joel.sch...@amd.com; > kim.phill...@freescale.com; pau...@samba.org; g...@kernel.org; > pbonz...@redhat.com; ag...@suse.de > Cc: linux-kernel@vger.kernel.org; patc...@linaro.org; will.dea...@arm.com; > a.mota...@virtualopensystems.com; a.r...@virtualopensystems.com; > john.li...@huawei.com; ming@canonical.com; Wu, Feng > Subject: [PATCH v3 0/8] KVM-VFIO IRQ forward control > > This series proposes an integration of "ARM: Forwarding physical > interrupts to a guest VM" (http://lwn.net/Articles/603514/) in > KVM. > > It enables to transform a VFIO platform driver IRQ into a forwarded > IRQ. > > When a physical IRQ is forwarded (to a guest), the host does not > deactivates this latter. Completion ownership is transferred to the > guest. When the guest deactivates the associated virtual IRQ, > the interrupt controler automatically completes the physical IRQ. > Obviously this requires some dedicated HW support in the interrupt > controler. > > The direct benefit is that, for a level sensitive IRQ, it avoids a > VM exit on forwarded IRQ completion. > > When the IRQ is forwarded, the VFIO platform driver does not need to > mask the physical IRQ anymore before signaling the eventfd. Indeed > genirq lowers the running priority, enabling other physical IRQ to hit > except that one. > > Besides, the injection still is based on irqfd triggering. The only > impact on irqfd process is resamplefd is not called anymore on > virtual IRQ completion since this latter becomes "transparent". > > The current integration is based on an extension of the KVM-VFIO > device, previously used by KVM to interact with VFIO groups. The > patch series now enables KVM to directly interact with a VFIO > platform device. The VFIO external API was extended for that purpose. > > Th KVM-VFIO device can get/put the vfio platform device, check its > integrity and type, get the IRQ number associated to an IRQ index. > > The IRQ forward programming is architecture specific (virtual interrupt > controller programming basically). However the whole infrastructure is > kept generic. > > from a user point of view, the functionality is provided through a > new KVM-VFIO group named KVM_DEV_VFIO_DEVICE and 2 associated > attributes: > - KVM_DEV_VFIO_DEVICE_FORWARD_IRQ, > - KVM_DEV_VFIO_DEVICE_UNFORWARD_IRQ. > > The capability can be checked with KVM_HAS_DEVICE_ATTR. > > Forwarding must be activated before VFIO signaling mechanism is set > using VFIO_DEVICE_SET_IRQS and unset while the signaling is disabled. > > --- > > This patch series has the following dependencies: > - "ARM: Forwarding physical interrupts to a guest VM" > (http://lwn.net/Articles/603514/) > - [PATCH v9 00/19] VFIO support for platform and AMBA devices on ARM > (http://www.spinics.net/lists/kvm-arm/msg11745.html) > - [PATCH v2 0/6] vfio: type1: support for ARM SMMUS with > VFIO_IOMMU_TYPE1 > (http://www.spinics.net/lists/kvm-arm/msg11738.html) > > Integrated pieces can be found at > ssh://git.linaro.org/people/eric.auger/linux.git > on branch irqfd_integ_v8 > > This was was tested on Calxeda Midway, assigning the xgmac main IRQ. > Hi Eric, Did you send out the latest QEMU part for this patch set, I notice that v6 of The QEMU part is sent out, but seems some structure in this new version has been changed, such as, struct kvm_arch_forwarded_irq (subindex is added in this version), so a new patchset in QEMU is also needed. Thanks, Feng > v2 -> v3: > - kvm_fwd_irq_action enum replaced by a bool (KVM_VFIO_IRQ_CLEANUP does > not > exist anymore) > - a new struct local to vfio.c was introduced to wrap kvm_fw_irq and make it > linkable: kvm_vfio_fwd_irq_node > - kvm_fwd_irq now is self-contained (includes struct vfio_device *) > - a single list of kvm_vfio_fwd_irq_irq_node is used instead of having > a list of devices and a list of forward irq per device. Having 2 lists > brought extra complexity. > - the VFIO device ref counter is incremented each time a new IRQ is forwarded. > It is not attempted anymore to hold a single reference whatever the number > of forwarded IRQs. > - subindex added on top of index to be closer to VFIO API > - platform device check moved in the arm specific implementation > - enable the KVM-VFIO device for arm64 > - forwarded state change only can happen while the VFIO IRQ handler is not > set; in other words, when the VFIO IRQ signaling is not set. > > v1 -> v2: > - forward control is moved from architecture specific file into generic > vfio.c module. > only kvm_arch_set_fwd_state remains architecture specific > - integrate Kim's patch which enables KVM-VFIO for ARM > - fix vgic
RE: [PATCH v3 0/8] KVM-VFIO IRQ forward control
-Original Message- From: Eric Auger [mailto:eric.au...@linaro.org] Sent: Monday, November 24, 2014 2:36 AM To: eric.au...@st.com; eric.au...@linaro.org; christoffer.d...@linaro.org; marc.zyng...@arm.com; linux-arm-ker...@lists.infradead.org; kvm...@lists.cs.columbia.edu; k...@vger.kernel.org; alex.william...@redhat.com; joel.sch...@amd.com; kim.phill...@freescale.com; pau...@samba.org; g...@kernel.org; pbonz...@redhat.com; ag...@suse.de Cc: linux-kernel@vger.kernel.org; patc...@linaro.org; will.dea...@arm.com; a.mota...@virtualopensystems.com; a.r...@virtualopensystems.com; john.li...@huawei.com; ming@canonical.com; Wu, Feng Subject: [PATCH v3 0/8] KVM-VFIO IRQ forward control This series proposes an integration of ARM: Forwarding physical interrupts to a guest VM (http://lwn.net/Articles/603514/) in KVM. It enables to transform a VFIO platform driver IRQ into a forwarded IRQ. When a physical IRQ is forwarded (to a guest), the host does not deactivates this latter. Completion ownership is transferred to the guest. When the guest deactivates the associated virtual IRQ, the interrupt controler automatically completes the physical IRQ. Obviously this requires some dedicated HW support in the interrupt controler. The direct benefit is that, for a level sensitive IRQ, it avoids a VM exit on forwarded IRQ completion. When the IRQ is forwarded, the VFIO platform driver does not need to mask the physical IRQ anymore before signaling the eventfd. Indeed genirq lowers the running priority, enabling other physical IRQ to hit except that one. Besides, the injection still is based on irqfd triggering. The only impact on irqfd process is resamplefd is not called anymore on virtual IRQ completion since this latter becomes transparent. The current integration is based on an extension of the KVM-VFIO device, previously used by KVM to interact with VFIO groups. The patch series now enables KVM to directly interact with a VFIO platform device. The VFIO external API was extended for that purpose. Th KVM-VFIO device can get/put the vfio platform device, check its integrity and type, get the IRQ number associated to an IRQ index. The IRQ forward programming is architecture specific (virtual interrupt controller programming basically). However the whole infrastructure is kept generic. from a user point of view, the functionality is provided through a new KVM-VFIO group named KVM_DEV_VFIO_DEVICE and 2 associated attributes: - KVM_DEV_VFIO_DEVICE_FORWARD_IRQ, - KVM_DEV_VFIO_DEVICE_UNFORWARD_IRQ. The capability can be checked with KVM_HAS_DEVICE_ATTR. Forwarding must be activated before VFIO signaling mechanism is set using VFIO_DEVICE_SET_IRQS and unset while the signaling is disabled. --- This patch series has the following dependencies: - ARM: Forwarding physical interrupts to a guest VM (http://lwn.net/Articles/603514/) - [PATCH v9 00/19] VFIO support for platform and AMBA devices on ARM (http://www.spinics.net/lists/kvm-arm/msg11745.html) - [PATCH v2 0/6] vfio: type1: support for ARM SMMUS with VFIO_IOMMU_TYPE1 (http://www.spinics.net/lists/kvm-arm/msg11738.html) Integrated pieces can be found at ssh://git.linaro.org/people/eric.auger/linux.git on branch irqfd_integ_v8 This was was tested on Calxeda Midway, assigning the xgmac main IRQ. Hi Eric, Did you send out the latest QEMU part for this patch set, I notice that v6 of The QEMU part is sent out, but seems some structure in this new version has been changed, such as, struct kvm_arch_forwarded_irq (subindex is added in this version), so a new patchset in QEMU is also needed. Thanks, Feng v2 - v3: - kvm_fwd_irq_action enum replaced by a bool (KVM_VFIO_IRQ_CLEANUP does not exist anymore) - a new struct local to vfio.c was introduced to wrap kvm_fw_irq and make it linkable: kvm_vfio_fwd_irq_node - kvm_fwd_irq now is self-contained (includes struct vfio_device *) - a single list of kvm_vfio_fwd_irq_irq_node is used instead of having a list of devices and a list of forward irq per device. Having 2 lists brought extra complexity. - the VFIO device ref counter is incremented each time a new IRQ is forwarded. It is not attempted anymore to hold a single reference whatever the number of forwarded IRQs. - subindex added on top of index to be closer to VFIO API - platform device check moved in the arm specific implementation - enable the KVM-VFIO device for arm64 - forwarded state change only can happen while the VFIO IRQ handler is not set; in other words, when the VFIO IRQ signaling is not set. v1 - v2: - forward control is moved from architecture specific file into generic vfio.c module. only kvm_arch_set_fwd_state remains architecture specific - integrate Kim's patch which enables KVM-VFIO for ARM - fix vgic state bypass in vgic_queue_hwirq - struct kvm_arch_forwarded_irq moved from arch/arm/include/uapi/asm/kvm.h
Re: [PATCH v3 0/8] KVM-VFIO IRQ forward control
On 11/24/2014 09:14 AM, Wu, Feng wrote: -Original Message- From: Eric Auger [mailto:eric.au...@linaro.org] Sent: Monday, November 24, 2014 2:36 AM To: eric.au...@st.com; eric.au...@linaro.org; christoffer.d...@linaro.org; marc.zyng...@arm.com; linux-arm-ker...@lists.infradead.org; kvm...@lists.cs.columbia.edu; k...@vger.kernel.org; alex.william...@redhat.com; joel.sch...@amd.com; kim.phill...@freescale.com; pau...@samba.org; g...@kernel.org; pbonz...@redhat.com; ag...@suse.de Cc: linux-kernel@vger.kernel.org; patc...@linaro.org; will.dea...@arm.com; a.mota...@virtualopensystems.com; a.r...@virtualopensystems.com; john.li...@huawei.com; ming@canonical.com; Wu, Feng Subject: [PATCH v3 0/8] KVM-VFIO IRQ forward control This series proposes an integration of ARM: Forwarding physical interrupts to a guest VM (http://lwn.net/Articles/603514/) in KVM. It enables to transform a VFIO platform driver IRQ into a forwarded IRQ. When a physical IRQ is forwarded (to a guest), the host does not deactivates this latter. Completion ownership is transferred to the guest. When the guest deactivates the associated virtual IRQ, the interrupt controler automatically completes the physical IRQ. Obviously this requires some dedicated HW support in the interrupt controler. The direct benefit is that, for a level sensitive IRQ, it avoids a VM exit on forwarded IRQ completion. When the IRQ is forwarded, the VFIO platform driver does not need to mask the physical IRQ anymore before signaling the eventfd. Indeed genirq lowers the running priority, enabling other physical IRQ to hit except that one. Besides, the injection still is based on irqfd triggering. The only impact on irqfd process is resamplefd is not called anymore on virtual IRQ completion since this latter becomes transparent. The current integration is based on an extension of the KVM-VFIO device, previously used by KVM to interact with VFIO groups. The patch series now enables KVM to directly interact with a VFIO platform device. The VFIO external API was extended for that purpose. Th KVM-VFIO device can get/put the vfio platform device, check its integrity and type, get the IRQ number associated to an IRQ index. The IRQ forward programming is architecture specific (virtual interrupt controller programming basically). However the whole infrastructure is kept generic. from a user point of view, the functionality is provided through a new KVM-VFIO group named KVM_DEV_VFIO_DEVICE and 2 associated attributes: - KVM_DEV_VFIO_DEVICE_FORWARD_IRQ, - KVM_DEV_VFIO_DEVICE_UNFORWARD_IRQ. The capability can be checked with KVM_HAS_DEVICE_ATTR. Forwarding must be activated before VFIO signaling mechanism is set using VFIO_DEVICE_SET_IRQS and unset while the signaling is disabled. --- This patch series has the following dependencies: - ARM: Forwarding physical interrupts to a guest VM (http://lwn.net/Articles/603514/) - [PATCH v9 00/19] VFIO support for platform and AMBA devices on ARM (http://www.spinics.net/lists/kvm-arm/msg11745.html) - [PATCH v2 0/6] vfio: type1: support for ARM SMMUS with VFIO_IOMMU_TYPE1 (http://www.spinics.net/lists/kvm-arm/msg11738.html) Integrated pieces can be found at ssh://git.linaro.org/people/eric.auger/linux.git on branch irqfd_integ_v8 This was was tested on Calxeda Midway, assigning the xgmac main IRQ. Hi Eric, Did you send out the latest QEMU part for this patch set, I notice that v6 of The QEMU part is sent out, but seems some structure in this new version has been changed, such as, struct kvm_arch_forwarded_irq (subindex is added in this version), so a new patchset in QEMU is also needed. Hi Feng, v7 is available at: http://lists.gnu.org/archive/html/qemu-devel/2014-10/msg03804.html. It already illustrates KVM-VFIO device usage. v8 which will indeed integrate subindex addition will be delivered this week. Best Regards Eric Thanks, Feng v2 - v3: - kvm_fwd_irq_action enum replaced by a bool (KVM_VFIO_IRQ_CLEANUP does not exist anymore) - a new struct local to vfio.c was introduced to wrap kvm_fw_irq and make it linkable: kvm_vfio_fwd_irq_node - kvm_fwd_irq now is self-contained (includes struct vfio_device *) - a single list of kvm_vfio_fwd_irq_irq_node is used instead of having a list of devices and a list of forward irq per device. Having 2 lists brought extra complexity. - the VFIO device ref counter is incremented each time a new IRQ is forwarded. It is not attempted anymore to hold a single reference whatever the number of forwarded IRQs. - subindex added on top of index to be closer to VFIO API - platform device check moved in the arm specific implementation - enable the KVM-VFIO device for arm64 - forwarded state change only can happen while the VFIO IRQ handler is not set; in other words, when the VFIO IRQ signaling is not set. v1 - v2: - forward control is moved from architecture
RE: [PATCH v3 0/8] KVM-VFIO IRQ forward control
-Original Message- From: Eric Auger [mailto:eric.au...@linaro.org] Sent: Monday, November 24, 2014 4:27 PM To: Wu, Feng; eric.au...@st.com; christoffer.d...@linaro.org; marc.zyng...@arm.com; linux-arm-ker...@lists.infradead.org; kvm...@lists.cs.columbia.edu; k...@vger.kernel.org; alex.william...@redhat.com; joel.sch...@amd.com; kim.phill...@freescale.com; pau...@samba.org; g...@kernel.org; pbonz...@redhat.com; ag...@suse.de Cc: linux-kernel@vger.kernel.org; patc...@linaro.org; will.dea...@arm.com; a.mota...@virtualopensystems.com; a.r...@virtualopensystems.com; john.li...@huawei.com; ming@canonical.com Subject: Re: [PATCH v3 0/8] KVM-VFIO IRQ forward control On 11/24/2014 09:14 AM, Wu, Feng wrote: -Original Message- From: Eric Auger [mailto:eric.au...@linaro.org] Sent: Monday, November 24, 2014 2:36 AM To: eric.au...@st.com; eric.au...@linaro.org; christoffer.d...@linaro.org; marc.zyng...@arm.com; linux-arm-ker...@lists.infradead.org; kvm...@lists.cs.columbia.edu; k...@vger.kernel.org; alex.william...@redhat.com; joel.sch...@amd.com; kim.phill...@freescale.com; pau...@samba.org; g...@kernel.org; pbonz...@redhat.com; ag...@suse.de Cc: linux-kernel@vger.kernel.org; patc...@linaro.org; will.dea...@arm.com; a.mota...@virtualopensystems.com; a.r...@virtualopensystems.com; john.li...@huawei.com; ming@canonical.com; Wu, Feng Subject: [PATCH v3 0/8] KVM-VFIO IRQ forward control This series proposes an integration of ARM: Forwarding physical interrupts to a guest VM (http://lwn.net/Articles/603514/) in KVM. It enables to transform a VFIO platform driver IRQ into a forwarded IRQ. When a physical IRQ is forwarded (to a guest), the host does not deactivates this latter. Completion ownership is transferred to the guest. When the guest deactivates the associated virtual IRQ, the interrupt controler automatically completes the physical IRQ. Obviously this requires some dedicated HW support in the interrupt controler. The direct benefit is that, for a level sensitive IRQ, it avoids a VM exit on forwarded IRQ completion. When the IRQ is forwarded, the VFIO platform driver does not need to mask the physical IRQ anymore before signaling the eventfd. Indeed genirq lowers the running priority, enabling other physical IRQ to hit except that one. Besides, the injection still is based on irqfd triggering. The only impact on irqfd process is resamplefd is not called anymore on virtual IRQ completion since this latter becomes transparent. The current integration is based on an extension of the KVM-VFIO device, previously used by KVM to interact with VFIO groups. The patch series now enables KVM to directly interact with a VFIO platform device. The VFIO external API was extended for that purpose. Th KVM-VFIO device can get/put the vfio platform device, check its integrity and type, get the IRQ number associated to an IRQ index. The IRQ forward programming is architecture specific (virtual interrupt controller programming basically). However the whole infrastructure is kept generic. from a user point of view, the functionality is provided through a new KVM-VFIO group named KVM_DEV_VFIO_DEVICE and 2 associated attributes: - KVM_DEV_VFIO_DEVICE_FORWARD_IRQ, - KVM_DEV_VFIO_DEVICE_UNFORWARD_IRQ. The capability can be checked with KVM_HAS_DEVICE_ATTR. Forwarding must be activated before VFIO signaling mechanism is set using VFIO_DEVICE_SET_IRQS and unset while the signaling is disabled. --- This patch series has the following dependencies: - ARM: Forwarding physical interrupts to a guest VM (http://lwn.net/Articles/603514/) - [PATCH v9 00/19] VFIO support for platform and AMBA devices on ARM (http://www.spinics.net/lists/kvm-arm/msg11745.html) - [PATCH v2 0/6] vfio: type1: support for ARM SMMUS with VFIO_IOMMU_TYPE1 (http://www.spinics.net/lists/kvm-arm/msg11738.html) Integrated pieces can be found at ssh://git.linaro.org/people/eric.auger/linux.git on branch irqfd_integ_v8 This was was tested on Calxeda Midway, assigning the xgmac main IRQ. Hi Eric, Did you send out the latest QEMU part for this patch set, I notice that v6 of The QEMU part is sent out, but seems some structure in this new version has been changed, such as, struct kvm_arch_forwarded_irq (subindex is added in this version), so a new patchset in QEMU is also needed. Hi Feng, v7 is available at: http://lists.gnu.org/archive/html/qemu-devel/2014-10/msg03804.html. It already illustrates KVM-VFIO device usage. v8 which will indeed integrate subindex addition will be delivered this week. Best Regards Eric Thanks a lot for the information, Eric! Thanks, Feng Thanks, Feng v2 - v3: - kvm_fwd_irq_action enum replaced by a bool (KVM_VFIO_IRQ_CLEANUP does not exist anymore) - a new struct local to vfio.c
