Re: [PATCH kernel 9/9] KVM: PPC: VFIO device: support SPAPR TCE

2016-06-14 Thread David Gibson 
On Tue, Jun 14, 2016 at 01:30:53PM +1000, Alexey Kardashevskiy wrote:
> On 10/06/16 16:50, David Gibson wrote:
> > On Thu, Jun 09, 2016 at 04:47:59PM +1000, Alexey Kardashevskiy wrote:
> >> On 23/03/16 14:03, David Gibson wrote:
> >>> On Tue, Mar 22, 2016 at 11:34:55AM +1100, Alexey Kardashevskiy wrote:
>  Uff, lost cc: list. Added back. Some comments below.
> 
> 
>  On 03/21/2016 04:19 PM, David Gibson wrote:
> > On Fri, Mar 18, 2016 at 11:12:26PM +1100, Alexey Kardashevskiy wrote:
> >> On March 15, 2016 17:29:26 David Gibson  
> >> wrote:
> >>
> >>> On Fri, Mar 11, 2016 at 10:09:50AM +1100, Alexey Kardashevskiy wrote:
>  On 03/10/2016 04:21 PM, David Gibson wrote:
> > On Wed, Mar 09, 2016 at 08:20:12PM +1100, Alexey Kardashevskiy 
> > wrote:
> >> On 03/09/2016 04:45 PM, David Gibson wrote:
> >>> On Mon, Mar 07, 2016 at 02:41:17PM +1100, Alexey Kardashevskiy 
> >>> wrote:
>  sPAPR TCE IOMMU is para-virtualized and the guest does map/unmap
>  via hypercalls which take a logical bus id (LIOBN) as a target 
>  IOMMU
>  identifier. LIOBNs are made up, advertised to guest systems and
>  linked to IOMMU groups by the user space.
>  In order to enable acceleration for IOMMU operations in KVM, we 
>  need
>  to tell KVM the information about the LIOBN-to-group mapping.
> 
>  For that, a new KVM_DEV_VFIO_GROUP_SET_SPAPR_TCE_LIOBN parameter
>  is added which accepts:
>  - a VFIO group fd and IO base address to find the actual hardware
>  TCE table;
>  - a LIOBN to assign to the found table.
> 
>  Before notifying KVM about new link, this check the group for 
>  being
>  registered with KVM device in order to release them at 
>  unexpected KVM
>  finish.
> 
>  This advertises the new KVM_CAP_SPAPR_TCE_VFIO capability to the 
>  user
>  space.
> 
>  While we are here, this also fixes VFIO KVM device compiling to 
>  let it
>  link to a KVM module.
> 
>  Signed-off-by: Alexey Kardashevskiy 
>  ---
>   Documentation/virtual/kvm/devices/vfio.txt |  21 +-
>   arch/powerpc/kvm/Kconfig   |   1 +
>   arch/powerpc/kvm/Makefile  |   5 +-
>   arch/powerpc/kvm/powerpc.c |   1 +
>   include/uapi/linux/kvm.h   |   9 +++
>   virt/kvm/vfio.c| 106
>  +
>   6 files changed, 140 insertions(+), 3 deletions(-)
> 
>  diff --git a/Documentation/virtual/kvm/devices/vfio.txt
>  b/Documentation/virtual/kvm/devices/vfio.txt
>  index ef51740..c0d3eb7 100644
>  --- a/Documentation/virtual/kvm/devices/vfio.txt
>  +++ b/Documentation/virtual/kvm/devices/vfio.txt
>  @@ -16,7 +16,24 @@ Groups:
> 
>   KVM_DEV_VFIO_GROUP attributes:
> KVM_DEV_VFIO_GROUP_ADD: Add a VFIO group to VFIO-KVM device 
>  tracking
>  +kvm_device_attr.addr points to an int32_t file 
>  descriptor
>  +for the VFIO group.
> >>>
> >>> AFAICT these changes are accurate for VFIO as it is already, in 
> >>> which
> >>> case it might be clearer to put them in a separate patch.
> >>>
> KVM_DEV_VFIO_GROUP_DEL: Remove a VFIO group from VFIO-KVM 
>  device
>  tracking
>  +kvm_device_attr.addr points to an int32_t file 
>  descriptor
>  +for the VFIO group.
> 
>  -For each, kvm_device_attr.addr points to an int32_t file 
>  descriptor
>  -for the VFIO group.
>  +  KVM_DEV_VFIO_GROUP_SET_SPAPR_TCE_LIOBN: sets a liobn for a 
>  VFIO group
>  +kvm_device_attr.addr points to a struct:
>  +struct kvm_vfio_spapr_tce_liobn {
>  +__u32   argsz;
>  +__s32   fd;
>  +__u32   liobn;
>  +__u8pad[4];
>  +__u64   start_addr;
>  +};
>  +where
>  +@argsz is the size of kvm_vfio_spapr_tce_liobn;
>  +@fd is a file descriptor for a VFIO group;
>  +@liobn is a

Re: [PATCH kernel 9/9] KVM: PPC: VFIO device: support SPAPR TCE

2016-06-13 Thread Alexey Kardashevskiy 
On 10/06/16 16:50, David Gibson wrote:
> On Thu, Jun 09, 2016 at 04:47:59PM +1000, Alexey Kardashevskiy wrote:
>> On 23/03/16 14:03, David Gibson wrote:
>>> On Tue, Mar 22, 2016 at 11:34:55AM +1100, Alexey Kardashevskiy wrote:
 Uff, lost cc: list. Added back. Some comments below.


 On 03/21/2016 04:19 PM, David Gibson wrote:
> On Fri, Mar 18, 2016 at 11:12:26PM +1100, Alexey Kardashevskiy wrote:
>> On March 15, 2016 17:29:26 David Gibson  
>> wrote:
>>
>>> On Fri, Mar 11, 2016 at 10:09:50AM +1100, Alexey Kardashevskiy wrote:
 On 03/10/2016 04:21 PM, David Gibson wrote:
> On Wed, Mar 09, 2016 at 08:20:12PM +1100, Alexey Kardashevskiy wrote:
>> On 03/09/2016 04:45 PM, David Gibson wrote:
>>> On Mon, Mar 07, 2016 at 02:41:17PM +1100, Alexey Kardashevskiy 
>>> wrote:
 sPAPR TCE IOMMU is para-virtualized and the guest does map/unmap
 via hypercalls which take a logical bus id (LIOBN) as a target 
 IOMMU
 identifier. LIOBNs are made up, advertised to guest systems and
 linked to IOMMU groups by the user space.
 In order to enable acceleration for IOMMU operations in KVM, we 
 need
 to tell KVM the information about the LIOBN-to-group mapping.

 For that, a new KVM_DEV_VFIO_GROUP_SET_SPAPR_TCE_LIOBN parameter
 is added which accepts:
 - a VFIO group fd and IO base address to find the actual hardware
 TCE table;
 - a LIOBN to assign to the found table.

 Before notifying KVM about new link, this check the group for being
 registered with KVM device in order to release them at unexpected 
 KVM
 finish.

 This advertises the new KVM_CAP_SPAPR_TCE_VFIO capability to the 
 user
 space.

 While we are here, this also fixes VFIO KVM device compiling to 
 let it
 link to a KVM module.

 Signed-off-by: Alexey Kardashevskiy 
 ---
  Documentation/virtual/kvm/devices/vfio.txt |  21 +-
  arch/powerpc/kvm/Kconfig   |   1 +
  arch/powerpc/kvm/Makefile  |   5 +-
  arch/powerpc/kvm/powerpc.c |   1 +
  include/uapi/linux/kvm.h   |   9 +++
  virt/kvm/vfio.c| 106
 +
  6 files changed, 140 insertions(+), 3 deletions(-)

 diff --git a/Documentation/virtual/kvm/devices/vfio.txt
 b/Documentation/virtual/kvm/devices/vfio.txt
 index ef51740..c0d3eb7 100644
 --- a/Documentation/virtual/kvm/devices/vfio.txt
 +++ b/Documentation/virtual/kvm/devices/vfio.txt
 @@ -16,7 +16,24 @@ Groups:

  KVM_DEV_VFIO_GROUP attributes:
KVM_DEV_VFIO_GROUP_ADD: Add a VFIO group to VFIO-KVM device 
 tracking
 +  kvm_device_attr.addr points to an int32_t file descriptor
 +  for the VFIO group.
>>>
>>> AFAICT these changes are accurate for VFIO as it is already, in 
>>> which
>>> case it might be clearer to put them in a separate patch.
>>>
KVM_DEV_VFIO_GROUP_DEL: Remove a VFIO group from VFIO-KVM device
 tracking
 +  kvm_device_attr.addr points to an int32_t file descriptor
 +  for the VFIO group.

 -For each, kvm_device_attr.addr points to an int32_t file 
 descriptor
 -for the VFIO group.
 +  KVM_DEV_VFIO_GROUP_SET_SPAPR_TCE_LIOBN: sets a liobn for a VFIO 
 group
 +  kvm_device_attr.addr points to a struct:
 +  struct kvm_vfio_spapr_tce_liobn {
 +  __u32   argsz;
 +  __s32   fd;
 +  __u32   liobn;
 +  __u8pad[4];
 +  __u64   start_addr;
 +  };
 +  where
 +  @argsz is the size of kvm_vfio_spapr_tce_liobn;
 +  @fd is a file descriptor for a VFIO group;
 +  @liobn is a logical bus id to be associated with the 
 group;
 +  @start_addr is a DMA window offset on the IO (PCI) bus
>>>
>>> For the cause of DDW and multiple windows, I'm assuming you can call
>>> this multiple times with different LIOBNs and the same IOMMU group?
>>
>>
>> Yes. It is called twice per each group (when DDW is activated) - for 
>> 32bit

Re: [PATCH kernel 9/9] KVM: PPC: VFIO device: support SPAPR TCE

2016-06-10 Thread David Gibson 
On Thu, Jun 09, 2016 at 04:47:59PM +1000, Alexey Kardashevskiy wrote:
> On 23/03/16 14:03, David Gibson wrote:
> > On Tue, Mar 22, 2016 at 11:34:55AM +1100, Alexey Kardashevskiy wrote:
> >> Uff, lost cc: list. Added back. Some comments below.
> >>
> >>
> >> On 03/21/2016 04:19 PM, David Gibson wrote:
> >>> On Fri, Mar 18, 2016 at 11:12:26PM +1100, Alexey Kardashevskiy wrote:
>  On March 15, 2016 17:29:26 David Gibson  
>  wrote:
> 
> > On Fri, Mar 11, 2016 at 10:09:50AM +1100, Alexey Kardashevskiy wrote:
> >> On 03/10/2016 04:21 PM, David Gibson wrote:
> >>> On Wed, Mar 09, 2016 at 08:20:12PM +1100, Alexey Kardashevskiy wrote:
>  On 03/09/2016 04:45 PM, David Gibson wrote:
> > On Mon, Mar 07, 2016 at 02:41:17PM +1100, Alexey Kardashevskiy 
> > wrote:
> >> sPAPR TCE IOMMU is para-virtualized and the guest does map/unmap
> >> via hypercalls which take a logical bus id (LIOBN) as a target 
> >> IOMMU
> >> identifier. LIOBNs are made up, advertised to guest systems and
> >> linked to IOMMU groups by the user space.
> >> In order to enable acceleration for IOMMU operations in KVM, we 
> >> need
> >> to tell KVM the information about the LIOBN-to-group mapping.
> >>
> >> For that, a new KVM_DEV_VFIO_GROUP_SET_SPAPR_TCE_LIOBN parameter
> >> is added which accepts:
> >> - a VFIO group fd and IO base address to find the actual hardware
> >> TCE table;
> >> - a LIOBN to assign to the found table.
> >>
> >> Before notifying KVM about new link, this check the group for being
> >> registered with KVM device in order to release them at unexpected 
> >> KVM
> >> finish.
> >>
> >> This advertises the new KVM_CAP_SPAPR_TCE_VFIO capability to the 
> >> user
> >> space.
> >>
> >> While we are here, this also fixes VFIO KVM device compiling to 
> >> let it
> >> link to a KVM module.
> >>
> >> Signed-off-by: Alexey Kardashevskiy 
> >> ---
> >>  Documentation/virtual/kvm/devices/vfio.txt |  21 +-
> >>  arch/powerpc/kvm/Kconfig   |   1 +
> >>  arch/powerpc/kvm/Makefile  |   5 +-
> >>  arch/powerpc/kvm/powerpc.c |   1 +
> >>  include/uapi/linux/kvm.h   |   9 +++
> >>  virt/kvm/vfio.c| 106
> >> +
> >>  6 files changed, 140 insertions(+), 3 deletions(-)
> >>
> >> diff --git a/Documentation/virtual/kvm/devices/vfio.txt
> >> b/Documentation/virtual/kvm/devices/vfio.txt
> >> index ef51740..c0d3eb7 100644
> >> --- a/Documentation/virtual/kvm/devices/vfio.txt
> >> +++ b/Documentation/virtual/kvm/devices/vfio.txt
> >> @@ -16,7 +16,24 @@ Groups:
> >>
> >>  KVM_DEV_VFIO_GROUP attributes:
> >>KVM_DEV_VFIO_GROUP_ADD: Add a VFIO group to VFIO-KVM device 
> >> tracking
> >> +  kvm_device_attr.addr points to an int32_t file descriptor
> >> +  for the VFIO group.
> >
> > AFAICT these changes are accurate for VFIO as it is already, in 
> > which
> > case it might be clearer to put them in a separate patch.
> >
> >>KVM_DEV_VFIO_GROUP_DEL: Remove a VFIO group from VFIO-KVM device
> >> tracking
> >> +  kvm_device_attr.addr points to an int32_t file descriptor
> >> +  for the VFIO group.
> >>
> >> -For each, kvm_device_attr.addr points to an int32_t file 
> >> descriptor
> >> -for the VFIO group.
> >> +  KVM_DEV_VFIO_GROUP_SET_SPAPR_TCE_LIOBN: sets a liobn for a VFIO 
> >> group
> >> +  kvm_device_attr.addr points to a struct:
> >> +  struct kvm_vfio_spapr_tce_liobn {
> >> +  __u32   argsz;
> >> +  __s32   fd;
> >> +  __u32   liobn;
> >> +  __u8pad[4];
> >> +  __u64   start_addr;
> >> +  };
> >> +  where
> >> +  @argsz is the size of kvm_vfio_spapr_tce_liobn;
> >> +  @fd is a file descriptor for a VFIO group;
> >> +  @liobn is a logical bus id to be associated with the 
> >> group;
> >> +  @start_addr is a DMA window offset on the IO (PCI) bus
> >
> > For the cause of DDW and multiple windows, I'm assuming you can call
> > this multiple times with different LIOBNs and the same IOMMU group?
> 
> 
>  Yes. It is called twice per each group (when DDW is activated) - for 
>  32bit
>  and 64bit windows, this is why

Re: [PATCH kernel 9/9] KVM: PPC: VFIO device: support SPAPR TCE

2016-06-09 Thread Alexey Kardashevskiy 
On 23/03/16 14:03, David Gibson wrote:
> On Tue, Mar 22, 2016 at 11:34:55AM +1100, Alexey Kardashevskiy wrote:
>> Uff, lost cc: list. Added back. Some comments below.
>>
>>
>> On 03/21/2016 04:19 PM, David Gibson wrote:
>>> On Fri, Mar 18, 2016 at 11:12:26PM +1100, Alexey Kardashevskiy wrote:
 On March 15, 2016 17:29:26 David Gibson  
 wrote:

> On Fri, Mar 11, 2016 at 10:09:50AM +1100, Alexey Kardashevskiy wrote:
>> On 03/10/2016 04:21 PM, David Gibson wrote:
>>> On Wed, Mar 09, 2016 at 08:20:12PM +1100, Alexey Kardashevskiy wrote:
 On 03/09/2016 04:45 PM, David Gibson wrote:
> On Mon, Mar 07, 2016 at 02:41:17PM +1100, Alexey Kardashevskiy wrote:
>> sPAPR TCE IOMMU is para-virtualized and the guest does map/unmap
>> via hypercalls which take a logical bus id (LIOBN) as a target IOMMU
>> identifier. LIOBNs are made up, advertised to guest systems and
>> linked to IOMMU groups by the user space.
>> In order to enable acceleration for IOMMU operations in KVM, we need
>> to tell KVM the information about the LIOBN-to-group mapping.
>>
>> For that, a new KVM_DEV_VFIO_GROUP_SET_SPAPR_TCE_LIOBN parameter
>> is added which accepts:
>> - a VFIO group fd and IO base address to find the actual hardware
>> TCE table;
>> - a LIOBN to assign to the found table.
>>
>> Before notifying KVM about new link, this check the group for being
>> registered with KVM device in order to release them at unexpected KVM
>> finish.
>>
>> This advertises the new KVM_CAP_SPAPR_TCE_VFIO capability to the user
>> space.
>>
>> While we are here, this also fixes VFIO KVM device compiling to let 
>> it
>> link to a KVM module.
>>
>> Signed-off-by: Alexey Kardashevskiy 
>> ---
>>  Documentation/virtual/kvm/devices/vfio.txt |  21 +-
>>  arch/powerpc/kvm/Kconfig   |   1 +
>>  arch/powerpc/kvm/Makefile  |   5 +-
>>  arch/powerpc/kvm/powerpc.c |   1 +
>>  include/uapi/linux/kvm.h   |   9 +++
>>  virt/kvm/vfio.c| 106
>> +
>>  6 files changed, 140 insertions(+), 3 deletions(-)
>>
>> diff --git a/Documentation/virtual/kvm/devices/vfio.txt
>> b/Documentation/virtual/kvm/devices/vfio.txt
>> index ef51740..c0d3eb7 100644
>> --- a/Documentation/virtual/kvm/devices/vfio.txt
>> +++ b/Documentation/virtual/kvm/devices/vfio.txt
>> @@ -16,7 +16,24 @@ Groups:
>>
>>  KVM_DEV_VFIO_GROUP attributes:
>>KVM_DEV_VFIO_GROUP_ADD: Add a VFIO group to VFIO-KVM device 
>> tracking
>> +kvm_device_attr.addr points to an int32_t file descriptor
>> +for the VFIO group.
>
> AFAICT these changes are accurate for VFIO as it is already, in which
> case it might be clearer to put them in a separate patch.
>
>>KVM_DEV_VFIO_GROUP_DEL: Remove a VFIO group from VFIO-KVM device
>> tracking
>> +kvm_device_attr.addr points to an int32_t file descriptor
>> +for the VFIO group.
>>
>> -For each, kvm_device_attr.addr points to an int32_t file descriptor
>> -for the VFIO group.
>> +  KVM_DEV_VFIO_GROUP_SET_SPAPR_TCE_LIOBN: sets a liobn for a VFIO 
>> group
>> +kvm_device_attr.addr points to a struct:
>> +struct kvm_vfio_spapr_tce_liobn {
>> +__u32   argsz;
>> +__s32   fd;
>> +__u32   liobn;
>> +__u8pad[4];
>> +__u64   start_addr;
>> +};
>> +where
>> +@argsz is the size of kvm_vfio_spapr_tce_liobn;
>> +@fd is a file descriptor for a VFIO group;
>> +@liobn is a logical bus id to be associated with the 
>> group;
>> +@start_addr is a DMA window offset on the IO (PCI) bus
>
> For the cause of DDW and multiple windows, I'm assuming you can call
> this multiple times with different LIOBNs and the same IOMMU group?


 Yes. It is called twice per each group (when DDW is activated) - for 
 32bit
 and 64bit windows, this is why @start_addr is there.


>> diff --git a/arch/powerpc/kvm/Kconfig b/arch/powerpc/kvm/Kconfig
>> index 1059846..dfa3488 100644
>> --- a/arch/powerpc/kvm/Kconfig
>> +++ b/arch/powerpc/kvm/Kconfig
>> @@ -65,6 +65,7 @@ config KVM_BOOK3S_64
>>  select KVM
>>

Re: [PATCH kernel 9/9] KVM: PPC: VFIO device: support SPAPR TCE

2016-04-10 Thread David Gibson 
On Fri, Apr 08, 2016 at 07:13:06PM +1000, Alexey Kardashevskiy wrote:
> On 03/09/2016 04:45 PM, David Gibson wrote:
> 
> >>diff --git a/arch/powerpc/kvm/Makefile b/arch/powerpc/kvm/Makefile
> >>index 7f7b6d8..71f577c 100644
> >>--- a/arch/powerpc/kvm/Makefile
> >>+++ b/arch/powerpc/kvm/Makefile
> >>@@ -8,7 +8,7 @@ ccflags-y := -Ivirt/kvm -Iarch/powerpc/kvm
> >>  KVM := ../../../virt/kvm
> >>
> >>  common-objs-y = $(KVM)/kvm_main.o $(KVM)/coalesced_mmio.o \
> >>-   $(KVM)/eventfd.o $(KVM)/vfio.o
> >>+   $(KVM)/eventfd.o
> >
> >Please don't disable the VFIO device for the non-book3s case.  I added
> >it (even though it didn't do anything until now) so that libvirt
> >wouldn't choke when it finds it's not available.  Obviously the new
> >ioctl needs to be only for the right IOMMU setup, but the device
> >itself should be available always.
> 
> 
> After having a closer look, the statement above does not enable VFIO KVM
> device on book3s but does for everything else:
> 
> 
> common-objs-$(CONFIG_KVM_VFIO) += $(KVM)/vfio.o
> [...]
> kvm-e500-objs := \
> $(common-objs-y) \
> [...]
> kvm-objs-$(CONFIG_KVM_E500V2) := $(kvm-e500-objs)
> [...]
> kvm-e500mc-objs := \
> $(common-objs-y) \
> [...]
> kvm-objs-$(CONFIG_KVM_E500MC) := $(kvm-e500mc-objs)
> [...]
> kvm-book3s_32-objs := \
> $(common-objs-y) \
> [...]
> kvm-objs-$(CONFIG_KVM_BOOK3S_32) := $(kvm-book3s_32-objs)
> 
> 
> This is becaise CONFIG_KVM_BOOK3S_64 does not use "common-objs-y":

Oh, good grief.

> kvm-objs-$(CONFIG_KVM_BOOK3S_64) := $(kvm-book3s_64-module-objs)
> 
> 
> So I will keep vfio.o in the "common-objs-y" list and add:
> 
> +kvm-book3s_64-objs-$(CONFIG_KVM_VFIO) += \
> + $(KVM)/vfio.o

Ok.

-- 
David Gibson| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson


signature.asc
Description: PGP signature
___
Linuxppc-dev mailing list
Linuxppc-dev@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/linuxppc-dev

Re: [PATCH kernel 9/9] KVM: PPC: VFIO device: support SPAPR TCE

2016-04-08 Thread Alexey Kardashevskiy 

On 03/09/2016 04:45 PM, David Gibson wrote:


diff --git a/arch/powerpc/kvm/Makefile b/arch/powerpc/kvm/Makefile
index 7f7b6d8..71f577c 100644
--- a/arch/powerpc/kvm/Makefile
+++ b/arch/powerpc/kvm/Makefile
@@ -8,7 +8,7 @@ ccflags-y := -Ivirt/kvm -Iarch/powerpc/kvm
  KVM := ../../../virt/kvm

  common-objs-y = $(KVM)/kvm_main.o $(KVM)/coalesced_mmio.o \
-   $(KVM)/eventfd.o $(KVM)/vfio.o
+   $(KVM)/eventfd.o


Please don't disable the VFIO device for the non-book3s case.  I added
it (even though it didn't do anything until now) so that libvirt
wouldn't choke when it finds it's not available.  Obviously the new
ioctl needs to be only for the right IOMMU setup, but the device
itself should be available always.



After having a closer look, the statement above does not enable VFIO KVM 
device on book3s but does for everything else:



common-objs-$(CONFIG_KVM_VFIO) += $(KVM)/vfio.o
[...]
kvm-e500-objs := \
$(common-objs-y) \
[...]
kvm-objs-$(CONFIG_KVM_E500V2) := $(kvm-e500-objs)
[...]
kvm-e500mc-objs := \
$(common-objs-y) \
[...]
kvm-objs-$(CONFIG_KVM_E500MC) := $(kvm-e500mc-objs)
[...]
kvm-book3s_32-objs := \
$(common-objs-y) \
[...]
kvm-objs-$(CONFIG_KVM_BOOK3S_32) := $(kvm-book3s_32-objs)


This is becaise CONFIG_KVM_BOOK3S_64 does not use "common-objs-y":

kvm-objs-$(CONFIG_KVM_BOOK3S_64) := $(kvm-book3s_64-module-objs)


So I will keep vfio.o in the "common-objs-y" list and add:

+kvm-book3s_64-objs-$(CONFIG_KVM_VFIO) += \
+   $(KVM)/vfio.o


--
Alexey
___
Linuxppc-dev mailing list
Linuxppc-dev@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/linuxppc-dev

Re: [PATCH kernel 9/9] KVM: PPC: VFIO device: support SPAPR TCE

2016-03-22 Thread David Gibson 
On Tue, Mar 22, 2016 at 11:34:55AM +1100, Alexey Kardashevskiy wrote:
> Uff, lost cc: list. Added back. Some comments below.
> 
> 
> On 03/21/2016 04:19 PM, David Gibson wrote:
> >On Fri, Mar 18, 2016 at 11:12:26PM +1100, Alexey Kardashevskiy wrote:
> >>On March 15, 2016 17:29:26 David Gibson  wrote:
> >>
> >>>On Fri, Mar 11, 2016 at 10:09:50AM +1100, Alexey Kardashevskiy wrote:
> On 03/10/2016 04:21 PM, David Gibson wrote:
> >On Wed, Mar 09, 2016 at 08:20:12PM +1100, Alexey Kardashevskiy wrote:
> >>On 03/09/2016 04:45 PM, David Gibson wrote:
> >>>On Mon, Mar 07, 2016 at 02:41:17PM +1100, Alexey Kardashevskiy wrote:
> sPAPR TCE IOMMU is para-virtualized and the guest does map/unmap
> via hypercalls which take a logical bus id (LIOBN) as a target IOMMU
> identifier. LIOBNs are made up, advertised to guest systems and
> linked to IOMMU groups by the user space.
> In order to enable acceleration for IOMMU operations in KVM, we need
> to tell KVM the information about the LIOBN-to-group mapping.
> 
> For that, a new KVM_DEV_VFIO_GROUP_SET_SPAPR_TCE_LIOBN parameter
> is added which accepts:
> - a VFIO group fd and IO base address to find the actual hardware
> TCE table;
> - a LIOBN to assign to the found table.
> 
> Before notifying KVM about new link, this check the group for being
> registered with KVM device in order to release them at unexpected KVM
> finish.
> 
> This advertises the new KVM_CAP_SPAPR_TCE_VFIO capability to the user
> space.
> 
> While we are here, this also fixes VFIO KVM device compiling to let it
> link to a KVM module.
> 
> Signed-off-by: Alexey Kardashevskiy 
> ---
>   Documentation/virtual/kvm/devices/vfio.txt |  21 +-
>   arch/powerpc/kvm/Kconfig   |   1 +
>   arch/powerpc/kvm/Makefile  |   5 +-
>   arch/powerpc/kvm/powerpc.c |   1 +
>   include/uapi/linux/kvm.h   |   9 +++
>   virt/kvm/vfio.c| 106
> +
>   6 files changed, 140 insertions(+), 3 deletions(-)
> 
> diff --git a/Documentation/virtual/kvm/devices/vfio.txt
> b/Documentation/virtual/kvm/devices/vfio.txt
> index ef51740..c0d3eb7 100644
> --- a/Documentation/virtual/kvm/devices/vfio.txt
> +++ b/Documentation/virtual/kvm/devices/vfio.txt
> @@ -16,7 +16,24 @@ Groups:
> 
>   KVM_DEV_VFIO_GROUP attributes:
> KVM_DEV_VFIO_GROUP_ADD: Add a VFIO group to VFIO-KVM device 
>  tracking
> + kvm_device_attr.addr points to an int32_t file descriptor
> + for the VFIO group.
> >>>
> >>>AFAICT these changes are accurate for VFIO as it is already, in which
> >>>case it might be clearer to put them in a separate patch.
> >>>
> KVM_DEV_VFIO_GROUP_DEL: Remove a VFIO group from VFIO-KVM device
> tracking
> + kvm_device_attr.addr points to an int32_t file descriptor
> + for the VFIO group.
> 
> -For each, kvm_device_attr.addr points to an int32_t file descriptor
> -for the VFIO group.
> +  KVM_DEV_VFIO_GROUP_SET_SPAPR_TCE_LIOBN: sets a liobn for a VFIO 
> group
> + kvm_device_attr.addr points to a struct:
> + struct kvm_vfio_spapr_tce_liobn {
> + __u32   argsz;
> + __s32   fd;
> + __u32   liobn;
> + __u8pad[4];
> + __u64   start_addr;
> + };
> + where
> + @argsz is the size of kvm_vfio_spapr_tce_liobn;
> + @fd is a file descriptor for a VFIO group;
> + @liobn is a logical bus id to be associated with the 
> group;
> + @start_addr is a DMA window offset on the IO (PCI) bus
> >>>
> >>>For the cause of DDW and multiple windows, I'm assuming you can call
> >>>this multiple times with different LIOBNs and the same IOMMU group?
> >>
> >>
> >>Yes. It is called twice per each group (when DDW is activated) - for 
> >>32bit
> >>and 64bit windows, this is why @start_addr is there.
> >>
> >>
> diff --git a/arch/powerpc/kvm/Kconfig b/arch/powerpc/kvm/Kconfig
> index 1059846..dfa3488 100644
> --- a/arch/powerpc/kvm/Kconfig
> +++ b/arch/powerpc/kvm/Kconfig
> @@ -65,6 +65,7 @@ config KVM_BOOK3S_64
>   select KVM
>   select KVM_BOOK3S_PR_POSSIBLE if !KVM_BOOK3S_HV_POSSIBLE
>   select SPAPR_TCE_IOMMU if

Re: [PATCH kernel 9/9] KVM: PPC: VFIO device: support SPAPR TCE

2016-03-21 Thread Alexey Kardashevskiy 

Uff, lost cc: list. Added back. Some comments below.


On 03/21/2016 04:19 PM, David Gibson wrote:

On Fri, Mar 18, 2016 at 11:12:26PM +1100, Alexey Kardashevskiy wrote:

On March 15, 2016 17:29:26 David Gibson  wrote:


On Fri, Mar 11, 2016 at 10:09:50AM +1100, Alexey Kardashevskiy wrote:

On 03/10/2016 04:21 PM, David Gibson wrote:

On Wed, Mar 09, 2016 at 08:20:12PM +1100, Alexey Kardashevskiy wrote:

On 03/09/2016 04:45 PM, David Gibson wrote:

On Mon, Mar 07, 2016 at 02:41:17PM +1100, Alexey Kardashevskiy wrote:

sPAPR TCE IOMMU is para-virtualized and the guest does map/unmap
via hypercalls which take a logical bus id (LIOBN) as a target IOMMU
identifier. LIOBNs are made up, advertised to guest systems and
linked to IOMMU groups by the user space.
In order to enable acceleration for IOMMU operations in KVM, we need
to tell KVM the information about the LIOBN-to-group mapping.

For that, a new KVM_DEV_VFIO_GROUP_SET_SPAPR_TCE_LIOBN parameter
is added which accepts:
- a VFIO group fd and IO base address to find the actual hardware
TCE table;
- a LIOBN to assign to the found table.

Before notifying KVM about new link, this check the group for being
registered with KVM device in order to release them at unexpected KVM
finish.

This advertises the new KVM_CAP_SPAPR_TCE_VFIO capability to the user
space.

While we are here, this also fixes VFIO KVM device compiling to let it
link to a KVM module.

Signed-off-by: Alexey Kardashevskiy 
---
  Documentation/virtual/kvm/devices/vfio.txt |  21 +-
  arch/powerpc/kvm/Kconfig   |   1 +
  arch/powerpc/kvm/Makefile  |   5 +-
  arch/powerpc/kvm/powerpc.c |   1 +
  include/uapi/linux/kvm.h   |   9 +++
  virt/kvm/vfio.c| 106

+

  6 files changed, 140 insertions(+), 3 deletions(-)

diff --git a/Documentation/virtual/kvm/devices/vfio.txt

b/Documentation/virtual/kvm/devices/vfio.txt

index ef51740..c0d3eb7 100644
--- a/Documentation/virtual/kvm/devices/vfio.txt
+++ b/Documentation/virtual/kvm/devices/vfio.txt
@@ -16,7 +16,24 @@ Groups:

  KVM_DEV_VFIO_GROUP attributes:
KVM_DEV_VFIO_GROUP_ADD: Add a VFIO group to VFIO-KVM device tracking
+   kvm_device_attr.addr points to an int32_t file descriptor
+   for the VFIO group.


AFAICT these changes are accurate for VFIO as it is already, in which
case it might be clearer to put them in a separate patch.


KVM_DEV_VFIO_GROUP_DEL: Remove a VFIO group from VFIO-KVM device

tracking

+   kvm_device_attr.addr points to an int32_t file descriptor
+   for the VFIO group.

-For each, kvm_device_attr.addr points to an int32_t file descriptor
-for the VFIO group.
+  KVM_DEV_VFIO_GROUP_SET_SPAPR_TCE_LIOBN: sets a liobn for a VFIO group
+   kvm_device_attr.addr points to a struct:
+   struct kvm_vfio_spapr_tce_liobn {
+   __u32   argsz;
+   __s32   fd;
+   __u32   liobn;
+   __u8pad[4];
+   __u64   start_addr;
+   };
+   where
+   @argsz is the size of kvm_vfio_spapr_tce_liobn;
+   @fd is a file descriptor for a VFIO group;
+   @liobn is a logical bus id to be associated with the group;
+   @start_addr is a DMA window offset on the IO (PCI) bus


For the cause of DDW and multiple windows, I'm assuming you can call
this multiple times with different LIOBNs and the same IOMMU group?



Yes. It is called twice per each group (when DDW is activated) - for 32bit
and 64bit windows, this is why @start_addr is there.



diff --git a/arch/powerpc/kvm/Kconfig b/arch/powerpc/kvm/Kconfig
index 1059846..dfa3488 100644
--- a/arch/powerpc/kvm/Kconfig
+++ b/arch/powerpc/kvm/Kconfig
@@ -65,6 +65,7 @@ config KVM_BOOK3S_64
select KVM
select KVM_BOOK3S_PR_POSSIBLE if !KVM_BOOK3S_HV_POSSIBLE
select SPAPR_TCE_IOMMU if IOMMU_SUPPORT
+   select KVM_VFIO if VFIO
---help---
  Support running unmodified book3s_64 and book3s_32 guest kernels
  in virtual machines on book3s_64 host processors.
diff --git a/arch/powerpc/kvm/Makefile b/arch/powerpc/kvm/Makefile
index 7f7b6d8..71f577c 100644
--- a/arch/powerpc/kvm/Makefile
+++ b/arch/powerpc/kvm/Makefile
@@ -8,7 +8,7 @@ ccflags-y := -Ivirt/kvm -Iarch/powerpc/kvm
  KVM := ../../../virt/kvm

  common-objs-y = $(KVM)/kvm_main.o $(KVM)/coalesced_mmio.o \
-   $(KVM)/eventfd.o $(KVM)/vfio.o
+   $(KVM)/eventfd.o


Please don't disable the VFIO device for the non-book3s case.  I added
it (even though it didn't do anything until now) so that libvirt
wouldn't choke when it finds it's not available.  Obviously the new
ioctl needs to be only for the right IOMMU setup, but the device
itself should be available always.


Ah. Ok, I'll fix this. I just wanted to be able to

Re: [PATCH kernel 9/9] KVM: PPC: VFIO device: support SPAPR TCE

2016-03-15 Thread David Gibson 
On Fri, Mar 11, 2016 at 10:09:50AM +1100, Alexey Kardashevskiy wrote:
> On 03/10/2016 04:21 PM, David Gibson wrote:
> >On Wed, Mar 09, 2016 at 08:20:12PM +1100, Alexey Kardashevskiy wrote:
> >>On 03/09/2016 04:45 PM, David Gibson wrote:
> >>>On Mon, Mar 07, 2016 at 02:41:17PM +1100, Alexey Kardashevskiy wrote:
> sPAPR TCE IOMMU is para-virtualized and the guest does map/unmap
> via hypercalls which take a logical bus id (LIOBN) as a target IOMMU
> identifier. LIOBNs are made up, advertised to guest systems and
> linked to IOMMU groups by the user space.
> In order to enable acceleration for IOMMU operations in KVM, we need
> to tell KVM the information about the LIOBN-to-group mapping.
> 
> For that, a new KVM_DEV_VFIO_GROUP_SET_SPAPR_TCE_LIOBN parameter
> is added which accepts:
> - a VFIO group fd and IO base address to find the actual hardware
> TCE table;
> - a LIOBN to assign to the found table.
> 
> Before notifying KVM about new link, this check the group for being
> registered with KVM device in order to release them at unexpected KVM
> finish.
> 
> This advertises the new KVM_CAP_SPAPR_TCE_VFIO capability to the user
> space.
> 
> While we are here, this also fixes VFIO KVM device compiling to let it
> link to a KVM module.
> 
> Signed-off-by: Alexey Kardashevskiy 
> ---
>   Documentation/virtual/kvm/devices/vfio.txt |  21 +-
>   arch/powerpc/kvm/Kconfig   |   1 +
>   arch/powerpc/kvm/Makefile  |   5 +-
>   arch/powerpc/kvm/powerpc.c |   1 +
>   include/uapi/linux/kvm.h   |   9 +++
>   virt/kvm/vfio.c| 106 
>  +
>   6 files changed, 140 insertions(+), 3 deletions(-)
> 
> diff --git a/Documentation/virtual/kvm/devices/vfio.txt 
> b/Documentation/virtual/kvm/devices/vfio.txt
> index ef51740..c0d3eb7 100644
> --- a/Documentation/virtual/kvm/devices/vfio.txt
> +++ b/Documentation/virtual/kvm/devices/vfio.txt
> @@ -16,7 +16,24 @@ Groups:
> 
>   KVM_DEV_VFIO_GROUP attributes:
> KVM_DEV_VFIO_GROUP_ADD: Add a VFIO group to VFIO-KVM device tracking
> + kvm_device_attr.addr points to an int32_t file descriptor
> + for the VFIO group.
> >>>
> >>>AFAICT these changes are accurate for VFIO as it is already, in which
> >>>case it might be clearer to put them in a separate patch.
> >>>
> KVM_DEV_VFIO_GROUP_DEL: Remove a VFIO group from VFIO-KVM device 
>  tracking
> + kvm_device_attr.addr points to an int32_t file descriptor
> + for the VFIO group.
> 
> -For each, kvm_device_attr.addr points to an int32_t file descriptor
> -for the VFIO group.
> +  KVM_DEV_VFIO_GROUP_SET_SPAPR_TCE_LIOBN: sets a liobn for a VFIO group
> + kvm_device_attr.addr points to a struct:
> + struct kvm_vfio_spapr_tce_liobn {
> + __u32   argsz;
> + __s32   fd;
> + __u32   liobn;
> + __u8pad[4];
> + __u64   start_addr;
> + };
> + where
> + @argsz is the size of kvm_vfio_spapr_tce_liobn;
> + @fd is a file descriptor for a VFIO group;
> + @liobn is a logical bus id to be associated with the group;
> + @start_addr is a DMA window offset on the IO (PCI) bus
> >>>
> >>>For the cause of DDW and multiple windows, I'm assuming you can call
> >>>this multiple times with different LIOBNs and the same IOMMU group?
> >>
> >>
> >>Yes. It is called twice per each group (when DDW is activated) - for 32bit
> >>and 64bit windows, this is why @start_addr is there.
> >>
> >>
> diff --git a/arch/powerpc/kvm/Kconfig b/arch/powerpc/kvm/Kconfig
> index 1059846..dfa3488 100644
> --- a/arch/powerpc/kvm/Kconfig
> +++ b/arch/powerpc/kvm/Kconfig
> @@ -65,6 +65,7 @@ config KVM_BOOK3S_64
>   select KVM
>   select KVM_BOOK3S_PR_POSSIBLE if !KVM_BOOK3S_HV_POSSIBLE
>   select SPAPR_TCE_IOMMU if IOMMU_SUPPORT
> + select KVM_VFIO if VFIO
>   ---help---
> Support running unmodified book3s_64 and book3s_32 guest 
>  kernels
> in virtual machines on book3s_64 host processors.
> diff --git a/arch/powerpc/kvm/Makefile b/arch/powerpc/kvm/Makefile
> index 7f7b6d8..71f577c 100644
> --- a/arch/powerpc/kvm/Makefile
> +++ b/arch/powerpc/kvm/Makefile
> @@ -8,7 +8,7 @@ ccflags-y := -Ivirt/kvm -Iarch/powerpc/kvm
>   KVM := ../../../virt/kvm
> 
>   common-objs-y = $(KVM)/kvm_main.o $(KVM)/coalesced_mmio.o \
> - $(KVM)/eventfd.o $(KVM)/vfio.o
> + $(KVM)/eventfd.o
> >>>
> >>>Please don't disable the VFIO device for the non-book3s case.  I added
> >>>it (even though it didn't do anything

Re: [PATCH kernel 9/9] KVM: PPC: VFIO device: support SPAPR TCE

2016-03-10 Thread Alexey Kardashevskiy 

On 03/10/2016 04:21 PM, David Gibson wrote:

On Wed, Mar 09, 2016 at 08:20:12PM +1100, Alexey Kardashevskiy wrote:

On 03/09/2016 04:45 PM, David Gibson wrote:

On Mon, Mar 07, 2016 at 02:41:17PM +1100, Alexey Kardashevskiy wrote:

sPAPR TCE IOMMU is para-virtualized and the guest does map/unmap
via hypercalls which take a logical bus id (LIOBN) as a target IOMMU
identifier. LIOBNs are made up, advertised to guest systems and
linked to IOMMU groups by the user space.
In order to enable acceleration for IOMMU operations in KVM, we need
to tell KVM the information about the LIOBN-to-group mapping.

For that, a new KVM_DEV_VFIO_GROUP_SET_SPAPR_TCE_LIOBN parameter
is added which accepts:
- a VFIO group fd and IO base address to find the actual hardware
TCE table;
- a LIOBN to assign to the found table.

Before notifying KVM about new link, this check the group for being
registered with KVM device in order to release them at unexpected KVM
finish.

This advertises the new KVM_CAP_SPAPR_TCE_VFIO capability to the user
space.

While we are here, this also fixes VFIO KVM device compiling to let it
link to a KVM module.

Signed-off-by: Alexey Kardashevskiy 
---
  Documentation/virtual/kvm/devices/vfio.txt |  21 +-
  arch/powerpc/kvm/Kconfig   |   1 +
  arch/powerpc/kvm/Makefile  |   5 +-
  arch/powerpc/kvm/powerpc.c |   1 +
  include/uapi/linux/kvm.h   |   9 +++
  virt/kvm/vfio.c| 106 +
  6 files changed, 140 insertions(+), 3 deletions(-)

diff --git a/Documentation/virtual/kvm/devices/vfio.txt 
b/Documentation/virtual/kvm/devices/vfio.txt
index ef51740..c0d3eb7 100644
--- a/Documentation/virtual/kvm/devices/vfio.txt
+++ b/Documentation/virtual/kvm/devices/vfio.txt
@@ -16,7 +16,24 @@ Groups:

  KVM_DEV_VFIO_GROUP attributes:
KVM_DEV_VFIO_GROUP_ADD: Add a VFIO group to VFIO-KVM device tracking
+   kvm_device_attr.addr points to an int32_t file descriptor
+   for the VFIO group.


AFAICT these changes are accurate for VFIO as it is already, in which
case it might be clearer to put them in a separate patch.


KVM_DEV_VFIO_GROUP_DEL: Remove a VFIO group from VFIO-KVM device tracking
+   kvm_device_attr.addr points to an int32_t file descriptor
+   for the VFIO group.

-For each, kvm_device_attr.addr points to an int32_t file descriptor
-for the VFIO group.
+  KVM_DEV_VFIO_GROUP_SET_SPAPR_TCE_LIOBN: sets a liobn for a VFIO group
+   kvm_device_attr.addr points to a struct:
+   struct kvm_vfio_spapr_tce_liobn {
+   __u32   argsz;
+   __s32   fd;
+   __u32   liobn;
+   __u8pad[4];
+   __u64   start_addr;
+   };
+   where
+   @argsz is the size of kvm_vfio_spapr_tce_liobn;
+   @fd is a file descriptor for a VFIO group;
+   @liobn is a logical bus id to be associated with the group;
+   @start_addr is a DMA window offset on the IO (PCI) bus


For the cause of DDW and multiple windows, I'm assuming you can call
this multiple times with different LIOBNs and the same IOMMU group?



Yes. It is called twice per each group (when DDW is activated) - for 32bit
and 64bit windows, this is why @start_addr is there.



diff --git a/arch/powerpc/kvm/Kconfig b/arch/powerpc/kvm/Kconfig
index 1059846..dfa3488 100644
--- a/arch/powerpc/kvm/Kconfig
+++ b/arch/powerpc/kvm/Kconfig
@@ -65,6 +65,7 @@ config KVM_BOOK3S_64
select KVM
select KVM_BOOK3S_PR_POSSIBLE if !KVM_BOOK3S_HV_POSSIBLE
select SPAPR_TCE_IOMMU if IOMMU_SUPPORT
+   select KVM_VFIO if VFIO
---help---
  Support running unmodified book3s_64 and book3s_32 guest kernels
  in virtual machines on book3s_64 host processors.
diff --git a/arch/powerpc/kvm/Makefile b/arch/powerpc/kvm/Makefile
index 7f7b6d8..71f577c 100644
--- a/arch/powerpc/kvm/Makefile
+++ b/arch/powerpc/kvm/Makefile
@@ -8,7 +8,7 @@ ccflags-y := -Ivirt/kvm -Iarch/powerpc/kvm
  KVM := ../../../virt/kvm

  common-objs-y = $(KVM)/kvm_main.o $(KVM)/coalesced_mmio.o \
-   $(KVM)/eventfd.o $(KVM)/vfio.o
+   $(KVM)/eventfd.o


Please don't disable the VFIO device for the non-book3s case.  I added
it (even though it didn't do anything until now) so that libvirt
wouldn't choke when it finds it's not available.  Obviously the new
ioctl needs to be only for the right IOMMU setup, but the device
itself should be available always.


Ah. Ok, I'll fix this. I just wanted to be able to compile kvm as a module.



  CFLAGS_e500_mmu.o := -I.
  CFLAGS_e500_mmu_host.o := -I.
@@ -87,6 +87,9 @@ endif
  kvm-book3s_64-objs-$(CONFIG_KVM_XICS) += \
book3s_xics.o

+kvm-book3s_64-objs-$(CONFIG_KVM_VFIO) += \
+   $(KVM)/vfio.o \
+
  kvm-book3s_64-module-objs += \
$(KVM)/kvm_main.o \

Re: [PATCH kernel 9/9] KVM: PPC: VFIO device: support SPAPR TCE

2016-03-09 Thread David Gibson 
On Wed, Mar 09, 2016 at 08:20:12PM +1100, Alexey Kardashevskiy wrote:
> On 03/09/2016 04:45 PM, David Gibson wrote:
> >On Mon, Mar 07, 2016 at 02:41:17PM +1100, Alexey Kardashevskiy wrote:
> >>sPAPR TCE IOMMU is para-virtualized and the guest does map/unmap
> >>via hypercalls which take a logical bus id (LIOBN) as a target IOMMU
> >>identifier. LIOBNs are made up, advertised to guest systems and
> >>linked to IOMMU groups by the user space.
> >>In order to enable acceleration for IOMMU operations in KVM, we need
> >>to tell KVM the information about the LIOBN-to-group mapping.
> >>
> >>For that, a new KVM_DEV_VFIO_GROUP_SET_SPAPR_TCE_LIOBN parameter
> >>is added which accepts:
> >>- a VFIO group fd and IO base address to find the actual hardware
> >>TCE table;
> >>- a LIOBN to assign to the found table.
> >>
> >>Before notifying KVM about new link, this check the group for being
> >>registered with KVM device in order to release them at unexpected KVM
> >>finish.
> >>
> >>This advertises the new KVM_CAP_SPAPR_TCE_VFIO capability to the user
> >>space.
> >>
> >>While we are here, this also fixes VFIO KVM device compiling to let it
> >>link to a KVM module.
> >>
> >>Signed-off-by: Alexey Kardashevskiy 
> >>---
> >>  Documentation/virtual/kvm/devices/vfio.txt |  21 +-
> >>  arch/powerpc/kvm/Kconfig   |   1 +
> >>  arch/powerpc/kvm/Makefile  |   5 +-
> >>  arch/powerpc/kvm/powerpc.c |   1 +
> >>  include/uapi/linux/kvm.h   |   9 +++
> >>  virt/kvm/vfio.c| 106 
> >> +
> >>  6 files changed, 140 insertions(+), 3 deletions(-)
> >>
> >>diff --git a/Documentation/virtual/kvm/devices/vfio.txt 
> >>b/Documentation/virtual/kvm/devices/vfio.txt
> >>index ef51740..c0d3eb7 100644
> >>--- a/Documentation/virtual/kvm/devices/vfio.txt
> >>+++ b/Documentation/virtual/kvm/devices/vfio.txt
> >>@@ -16,7 +16,24 @@ Groups:
> >>
> >>  KVM_DEV_VFIO_GROUP attributes:
> >>KVM_DEV_VFIO_GROUP_ADD: Add a VFIO group to VFIO-KVM device tracking
> >>+   kvm_device_attr.addr points to an int32_t file descriptor
> >>+   for the VFIO group.
> >
> >AFAICT these changes are accurate for VFIO as it is already, in which
> >case it might be clearer to put them in a separate patch.
> >
> >>KVM_DEV_VFIO_GROUP_DEL: Remove a VFIO group from VFIO-KVM device 
> >> tracking
> >>+   kvm_device_attr.addr points to an int32_t file descriptor
> >>+   for the VFIO group.
> >>
> >>-For each, kvm_device_attr.addr points to an int32_t file descriptor
> >>-for the VFIO group.
> >>+  KVM_DEV_VFIO_GROUP_SET_SPAPR_TCE_LIOBN: sets a liobn for a VFIO group
> >>+   kvm_device_attr.addr points to a struct:
> >>+   struct kvm_vfio_spapr_tce_liobn {
> >>+   __u32   argsz;
> >>+   __s32   fd;
> >>+   __u32   liobn;
> >>+   __u8pad[4];
> >>+   __u64   start_addr;
> >>+   };
> >>+   where
> >>+   @argsz is the size of kvm_vfio_spapr_tce_liobn;
> >>+   @fd is a file descriptor for a VFIO group;
> >>+   @liobn is a logical bus id to be associated with the group;
> >>+   @start_addr is a DMA window offset on the IO (PCI) bus
> >
> >For the cause of DDW and multiple windows, I'm assuming you can call
> >this multiple times with different LIOBNs and the same IOMMU group?
> 
> 
> Yes. It is called twice per each group (when DDW is activated) - for 32bit
> and 64bit windows, this is why @start_addr is there.
> 
> 
> >>diff --git a/arch/powerpc/kvm/Kconfig b/arch/powerpc/kvm/Kconfig
> >>index 1059846..dfa3488 100644
> >>--- a/arch/powerpc/kvm/Kconfig
> >>+++ b/arch/powerpc/kvm/Kconfig
> >>@@ -65,6 +65,7 @@ config KVM_BOOK3S_64
> >>select KVM
> >>select KVM_BOOK3S_PR_POSSIBLE if !KVM_BOOK3S_HV_POSSIBLE
> >>select SPAPR_TCE_IOMMU if IOMMU_SUPPORT
> >>+   select KVM_VFIO if VFIO
> >>---help---
> >>  Support running unmodified book3s_64 and book3s_32 guest kernels
> >>  in virtual machines on book3s_64 host processors.
> >>diff --git a/arch/powerpc/kvm/Makefile b/arch/powerpc/kvm/Makefile
> >>index 7f7b6d8..71f577c 100644
> >>--- a/arch/powerpc/kvm/Makefile
> >>+++ b/arch/powerpc/kvm/Makefile
> >>@@ -8,7 +8,7 @@ ccflags-y := -Ivirt/kvm -Iarch/powerpc/kvm
> >>  KVM := ../../../virt/kvm
> >>
> >>  common-objs-y = $(KVM)/kvm_main.o $(KVM)/coalesced_mmio.o \
> >>-   $(KVM)/eventfd.o $(KVM)/vfio.o
> >>+   $(KVM)/eventfd.o
> >
> >Please don't disable the VFIO device for the non-book3s case.  I added
> >it (even though it didn't do anything until now) so that libvirt
> >wouldn't choke when it finds it's not available.  Obviously the new
> >ioctl needs to be only for the right IOMMU setup, but the device
> >itself should be available always.
> 
> Ah. Ok, I'll fix this. I just wanted to be able to compile kvm as a module.
> 
> 
> >>  CFLAGS_e500_mmu.o := -I.
> >>

Re: [PATCH kernel 9/9] KVM: PPC: VFIO device: support SPAPR TCE

2016-03-09 Thread Alexey Kardashevskiy 

On 03/09/2016 04:45 PM, David Gibson wrote:

On Mon, Mar 07, 2016 at 02:41:17PM +1100, Alexey Kardashevskiy wrote:

sPAPR TCE IOMMU is para-virtualized and the guest does map/unmap
via hypercalls which take a logical bus id (LIOBN) as a target IOMMU
identifier. LIOBNs are made up, advertised to guest systems and
linked to IOMMU groups by the user space.
In order to enable acceleration for IOMMU operations in KVM, we need
to tell KVM the information about the LIOBN-to-group mapping.

For that, a new KVM_DEV_VFIO_GROUP_SET_SPAPR_TCE_LIOBN parameter
is added which accepts:
- a VFIO group fd and IO base address to find the actual hardware
TCE table;
- a LIOBN to assign to the found table.

Before notifying KVM about new link, this check the group for being
registered with KVM device in order to release them at unexpected KVM
finish.

This advertises the new KVM_CAP_SPAPR_TCE_VFIO capability to the user
space.

While we are here, this also fixes VFIO KVM device compiling to let it
link to a KVM module.

Signed-off-by: Alexey Kardashevskiy 
---
  Documentation/virtual/kvm/devices/vfio.txt |  21 +-
  arch/powerpc/kvm/Kconfig   |   1 +
  arch/powerpc/kvm/Makefile  |   5 +-
  arch/powerpc/kvm/powerpc.c |   1 +
  include/uapi/linux/kvm.h   |   9 +++
  virt/kvm/vfio.c| 106 +
  6 files changed, 140 insertions(+), 3 deletions(-)

diff --git a/Documentation/virtual/kvm/devices/vfio.txt 
b/Documentation/virtual/kvm/devices/vfio.txt
index ef51740..c0d3eb7 100644
--- a/Documentation/virtual/kvm/devices/vfio.txt
+++ b/Documentation/virtual/kvm/devices/vfio.txt
@@ -16,7 +16,24 @@ Groups:

  KVM_DEV_VFIO_GROUP attributes:
KVM_DEV_VFIO_GROUP_ADD: Add a VFIO group to VFIO-KVM device tracking
+   kvm_device_attr.addr points to an int32_t file descriptor
+   for the VFIO group.


AFAICT these changes are accurate for VFIO as it is already, in which
case it might be clearer to put them in a separate patch.


KVM_DEV_VFIO_GROUP_DEL: Remove a VFIO group from VFIO-KVM device tracking
+   kvm_device_attr.addr points to an int32_t file descriptor
+   for the VFIO group.

-For each, kvm_device_attr.addr points to an int32_t file descriptor
-for the VFIO group.
+  KVM_DEV_VFIO_GROUP_SET_SPAPR_TCE_LIOBN: sets a liobn for a VFIO group
+   kvm_device_attr.addr points to a struct:
+   struct kvm_vfio_spapr_tce_liobn {
+   __u32   argsz;
+   __s32   fd;
+   __u32   liobn;
+   __u8pad[4];
+   __u64   start_addr;
+   };
+   where
+   @argsz is the size of kvm_vfio_spapr_tce_liobn;
+   @fd is a file descriptor for a VFIO group;
+   @liobn is a logical bus id to be associated with the group;
+   @start_addr is a DMA window offset on the IO (PCI) bus


For the cause of DDW and multiple windows, I'm assuming you can call
this multiple times with different LIOBNs and the same IOMMU group?



Yes. It is called twice per each group (when DDW is activated) - for 32bit 
and 64bit windows, this is why @start_addr is there.




diff --git a/arch/powerpc/kvm/Kconfig b/arch/powerpc/kvm/Kconfig
index 1059846..dfa3488 100644
--- a/arch/powerpc/kvm/Kconfig
+++ b/arch/powerpc/kvm/Kconfig
@@ -65,6 +65,7 @@ config KVM_BOOK3S_64
select KVM
select KVM_BOOK3S_PR_POSSIBLE if !KVM_BOOK3S_HV_POSSIBLE
select SPAPR_TCE_IOMMU if IOMMU_SUPPORT
+   select KVM_VFIO if VFIO
---help---
  Support running unmodified book3s_64 and book3s_32 guest kernels
  in virtual machines on book3s_64 host processors.
diff --git a/arch/powerpc/kvm/Makefile b/arch/powerpc/kvm/Makefile
index 7f7b6d8..71f577c 100644
--- a/arch/powerpc/kvm/Makefile
+++ b/arch/powerpc/kvm/Makefile
@@ -8,7 +8,7 @@ ccflags-y := -Ivirt/kvm -Iarch/powerpc/kvm
  KVM := ../../../virt/kvm

  common-objs-y = $(KVM)/kvm_main.o $(KVM)/coalesced_mmio.o \
-   $(KVM)/eventfd.o $(KVM)/vfio.o
+   $(KVM)/eventfd.o


Please don't disable the VFIO device for the non-book3s case.  I added
it (even though it didn't do anything until now) so that libvirt
wouldn't choke when it finds it's not available.  Obviously the new
ioctl needs to be only for the right IOMMU setup, but the device
itself should be available always.


Ah. Ok, I'll fix this. I just wanted to be able to compile kvm as a module.



  CFLAGS_e500_mmu.o := -I.
  CFLAGS_e500_mmu_host.o := -I.
@@ -87,6 +87,9 @@ endif
  kvm-book3s_64-objs-$(CONFIG_KVM_XICS) += \
book3s_xics.o

+kvm-book3s_64-objs-$(CONFIG_KVM_VFIO) += \
+   $(KVM)/vfio.o \
+
  kvm-book3s_64-module-objs += \
$(KVM)/kvm_main.o \
$(KVM)/eventfd.o \
diff --git a/arch/powerpc/kvm/powerpc.c b/arch/powerpc/kvm/powerpc.c
index 19aa59b..63f188d 100644
---

Re: [PATCH kernel 9/9] KVM: PPC: VFIO device: support SPAPR TCE

2016-03-08 Thread David Gibson 
On Mon, Mar 07, 2016 at 02:41:17PM +1100, Alexey Kardashevskiy wrote:
> sPAPR TCE IOMMU is para-virtualized and the guest does map/unmap
> via hypercalls which take a logical bus id (LIOBN) as a target IOMMU
> identifier. LIOBNs are made up, advertised to guest systems and
> linked to IOMMU groups by the user space.
> In order to enable acceleration for IOMMU operations in KVM, we need
> to tell KVM the information about the LIOBN-to-group mapping.
> 
> For that, a new KVM_DEV_VFIO_GROUP_SET_SPAPR_TCE_LIOBN parameter
> is added which accepts:
> - a VFIO group fd and IO base address to find the actual hardware
> TCE table;
> - a LIOBN to assign to the found table.
> 
> Before notifying KVM about new link, this check the group for being
> registered with KVM device in order to release them at unexpected KVM
> finish.
> 
> This advertises the new KVM_CAP_SPAPR_TCE_VFIO capability to the user
> space.
> 
> While we are here, this also fixes VFIO KVM device compiling to let it
> link to a KVM module.
> 
> Signed-off-by: Alexey Kardashevskiy 
> ---
>  Documentation/virtual/kvm/devices/vfio.txt |  21 +-
>  arch/powerpc/kvm/Kconfig   |   1 +
>  arch/powerpc/kvm/Makefile  |   5 +-
>  arch/powerpc/kvm/powerpc.c |   1 +
>  include/uapi/linux/kvm.h   |   9 +++
>  virt/kvm/vfio.c| 106 
> +
>  6 files changed, 140 insertions(+), 3 deletions(-)
> 
> diff --git a/Documentation/virtual/kvm/devices/vfio.txt 
> b/Documentation/virtual/kvm/devices/vfio.txt
> index ef51740..c0d3eb7 100644
> --- a/Documentation/virtual/kvm/devices/vfio.txt
> +++ b/Documentation/virtual/kvm/devices/vfio.txt
> @@ -16,7 +16,24 @@ Groups:
>  
>  KVM_DEV_VFIO_GROUP attributes:
>KVM_DEV_VFIO_GROUP_ADD: Add a VFIO group to VFIO-KVM device tracking
> + kvm_device_attr.addr points to an int32_t file descriptor
> + for the VFIO group.

AFAICT these changes are accurate for VFIO as it is already, in which
case it might be clearer to put them in a separate patch.

>KVM_DEV_VFIO_GROUP_DEL: Remove a VFIO group from VFIO-KVM device tracking
> + kvm_device_attr.addr points to an int32_t file descriptor
> + for the VFIO group.
>  
> -For each, kvm_device_attr.addr points to an int32_t file descriptor
> -for the VFIO group.
> +  KVM_DEV_VFIO_GROUP_SET_SPAPR_TCE_LIOBN: sets a liobn for a VFIO group
> + kvm_device_attr.addr points to a struct:
> + struct kvm_vfio_spapr_tce_liobn {
> + __u32   argsz;
> + __s32   fd;
> + __u32   liobn;
> + __u8pad[4];
> + __u64   start_addr;
> + };
> + where
> + @argsz is the size of kvm_vfio_spapr_tce_liobn;
> + @fd is a file descriptor for a VFIO group;
> + @liobn is a logical bus id to be associated with the group;
> + @start_addr is a DMA window offset on the IO (PCI) bus

For the cause of DDW and multiple windows, I'm assuming you can call
this multiple times with different LIOBNs and the same IOMMU group?

> diff --git a/arch/powerpc/kvm/Kconfig b/arch/powerpc/kvm/Kconfig
> index 1059846..dfa3488 100644
> --- a/arch/powerpc/kvm/Kconfig
> +++ b/arch/powerpc/kvm/Kconfig
> @@ -65,6 +65,7 @@ config KVM_BOOK3S_64
>   select KVM
>   select KVM_BOOK3S_PR_POSSIBLE if !KVM_BOOK3S_HV_POSSIBLE
>   select SPAPR_TCE_IOMMU if IOMMU_SUPPORT
> + select KVM_VFIO if VFIO
>   ---help---
> Support running unmodified book3s_64 and book3s_32 guest kernels
> in virtual machines on book3s_64 host processors.
> diff --git a/arch/powerpc/kvm/Makefile b/arch/powerpc/kvm/Makefile
> index 7f7b6d8..71f577c 100644
> --- a/arch/powerpc/kvm/Makefile
> +++ b/arch/powerpc/kvm/Makefile
> @@ -8,7 +8,7 @@ ccflags-y := -Ivirt/kvm -Iarch/powerpc/kvm
>  KVM := ../../../virt/kvm
>  
>  common-objs-y = $(KVM)/kvm_main.o $(KVM)/coalesced_mmio.o \
> - $(KVM)/eventfd.o $(KVM)/vfio.o
> + $(KVM)/eventfd.o

Please don't disable the VFIO device for the non-book3s case.  I added
it (even though it didn't do anything until now) so that libvirt
wouldn't choke when it finds it's not available.  Obviously the new
ioctl needs to be only for the right IOMMU setup, but the device
itself should be available always.

>  CFLAGS_e500_mmu.o := -I.
>  CFLAGS_e500_mmu_host.o := -I.
> @@ -87,6 +87,9 @@ endif
>  kvm-book3s_64-objs-$(CONFIG_KVM_XICS) += \
>   book3s_xics.o
>  
> +kvm-book3s_64-objs-$(CONFIG_KVM_VFIO) += \
> + $(KVM)/vfio.o \
> +
>  kvm-book3s_64-module-objs += \
>   $(KVM)/kvm_main.o \
>   $(KVM)/eventfd.o \
> diff --git a/arch/powerpc/kvm/powerpc.c b/arch/powerpc/kvm/powerpc.c
> index 19aa59b..63f188d 100644
> --- a/arch/powerpc/kvm/powerpc.c
> +++ b/arch/powerpc/kvm/powerpc.c
> @@ -521,6 +521,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm,

[PATCH kernel 9/9] KVM: PPC: VFIO device: support SPAPR TCE

2016-03-06 Thread Alexey Kardashevskiy 
sPAPR TCE IOMMU is para-virtualized and the guest does map/unmap
via hypercalls which take a logical bus id (LIOBN) as a target IOMMU
identifier. LIOBNs are made up, advertised to guest systems and
linked to IOMMU groups by the user space.
In order to enable acceleration for IOMMU operations in KVM, we need
to tell KVM the information about the LIOBN-to-group mapping.

For that, a new KVM_DEV_VFIO_GROUP_SET_SPAPR_TCE_LIOBN parameter
is added which accepts:
- a VFIO group fd and IO base address to find the actual hardware
TCE table;
- a LIOBN to assign to the found table.

Before notifying KVM about new link, this check the group for being
registered with KVM device in order to release them at unexpected KVM
finish.

This advertises the new KVM_CAP_SPAPR_TCE_VFIO capability to the user
space.

While we are here, this also fixes VFIO KVM device compiling to let it
link to a KVM module.

Signed-off-by: Alexey Kardashevskiy 
---
 Documentation/virtual/kvm/devices/vfio.txt |  21 +-
 arch/powerpc/kvm/Kconfig   |   1 +
 arch/powerpc/kvm/Makefile  |   5 +-
 arch/powerpc/kvm/powerpc.c |   1 +
 include/uapi/linux/kvm.h   |   9 +++
 virt/kvm/vfio.c| 106 +
 6 files changed, 140 insertions(+), 3 deletions(-)

diff --git a/Documentation/virtual/kvm/devices/vfio.txt 
b/Documentation/virtual/kvm/devices/vfio.txt
index ef51740..c0d3eb7 100644
--- a/Documentation/virtual/kvm/devices/vfio.txt
+++ b/Documentation/virtual/kvm/devices/vfio.txt
@@ -16,7 +16,24 @@ Groups:
 
 KVM_DEV_VFIO_GROUP attributes:
   KVM_DEV_VFIO_GROUP_ADD: Add a VFIO group to VFIO-KVM device tracking
+   kvm_device_attr.addr points to an int32_t file descriptor
+   for the VFIO group.
+
   KVM_DEV_VFIO_GROUP_DEL: Remove a VFIO group from VFIO-KVM device tracking
+   kvm_device_attr.addr points to an int32_t file descriptor
+   for the VFIO group.
 
-For each, kvm_device_attr.addr points to an int32_t file descriptor
-for the VFIO group.
+  KVM_DEV_VFIO_GROUP_SET_SPAPR_TCE_LIOBN: sets a liobn for a VFIO group
+   kvm_device_attr.addr points to a struct:
+   struct kvm_vfio_spapr_tce_liobn {
+   __u32   argsz;
+   __s32   fd;
+   __u32   liobn;
+   __u8pad[4];
+   __u64   start_addr;
+   };
+   where
+   @argsz is the size of kvm_vfio_spapr_tce_liobn;
+   @fd is a file descriptor for a VFIO group;
+   @liobn is a logical bus id to be associated with the group;
+   @start_addr is a DMA window offset on the IO (PCI) bus
diff --git a/arch/powerpc/kvm/Kconfig b/arch/powerpc/kvm/Kconfig
index 1059846..dfa3488 100644
--- a/arch/powerpc/kvm/Kconfig
+++ b/arch/powerpc/kvm/Kconfig
@@ -65,6 +65,7 @@ config KVM_BOOK3S_64
select KVM
select KVM_BOOK3S_PR_POSSIBLE if !KVM_BOOK3S_HV_POSSIBLE
select SPAPR_TCE_IOMMU if IOMMU_SUPPORT
+   select KVM_VFIO if VFIO
---help---
  Support running unmodified book3s_64 and book3s_32 guest kernels
  in virtual machines on book3s_64 host processors.
diff --git a/arch/powerpc/kvm/Makefile b/arch/powerpc/kvm/Makefile
index 7f7b6d8..71f577c 100644
--- a/arch/powerpc/kvm/Makefile
+++ b/arch/powerpc/kvm/Makefile
@@ -8,7 +8,7 @@ ccflags-y := -Ivirt/kvm -Iarch/powerpc/kvm
 KVM := ../../../virt/kvm
 
 common-objs-y = $(KVM)/kvm_main.o $(KVM)/coalesced_mmio.o \
-   $(KVM)/eventfd.o $(KVM)/vfio.o
+   $(KVM)/eventfd.o
 
 CFLAGS_e500_mmu.o := -I.
 CFLAGS_e500_mmu_host.o := -I.
@@ -87,6 +87,9 @@ endif
 kvm-book3s_64-objs-$(CONFIG_KVM_XICS) += \
book3s_xics.o
 
+kvm-book3s_64-objs-$(CONFIG_KVM_VFIO) += \
+   $(KVM)/vfio.o \
+
 kvm-book3s_64-module-objs += \
$(KVM)/kvm_main.o \
$(KVM)/eventfd.o \
diff --git a/arch/powerpc/kvm/powerpc.c b/arch/powerpc/kvm/powerpc.c
index 19aa59b..63f188d 100644
--- a/arch/powerpc/kvm/powerpc.c
+++ b/arch/powerpc/kvm/powerpc.c
@@ -521,6 +521,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
 #ifdef CONFIG_PPC_BOOK3S_64
case KVM_CAP_SPAPR_TCE:
case KVM_CAP_SPAPR_TCE_64:
+   case KVM_CAP_SPAPR_TCE_VFIO:
case KVM_CAP_PPC_ALLOC_HTAB:
case KVM_CAP_PPC_RTAS:
case KVM_CAP_PPC_FIXUP_HCALL:
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index 080ffbf..f1abbea 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -1056,6 +1056,7 @@ struct kvm_device_attr {
 #define  KVM_DEV_VFIO_GROUP1
 #define   KVM_DEV_VFIO_GROUP_ADD   1
 #define   KVM_DEV_VFIO_GROUP_DEL   2
+#define   KVM_DEV_VFIO_GROUP_SET_SPAPR_TCE_LIOBN   3
 
 enum kvm_device_type {
KVM_DEV_TYPE_FSL_MPIC_20= 1,
@@ -1075,6 +1076,14 @@ enum kvm_device_type {