Re: [PATCH RFC 11/12] iommufd: vfio container FD ioctl compatibility

2022-05-24 Thread David Gibson
o swap, or you call GUP for each vIOMMU > hypercall. > > Since everyone says PPC doesn't call GUP during the hypercall - how is > it working? The current implementation does GUP during the pre-reserve. I think Alexey's talking about a new PowerVM (IBM hypervisor) feat

Re: [PATCH RFC 11/12] iommufd: vfio container FD ioctl compatibility

2022-05-12 Thread David Gibson
On Wed, May 11, 2022 at 03:15:22AM +, Tian, Kevin wrote: > > From: Jason Gunthorpe > > Sent: Wednesday, May 11, 2022 3:00 AM > > > > On Tue, May 10, 2022 at 05:12:04PM +1000, David Gibson wrote: > > > Ok... here's a revised version of my proposal which I th

Re: [PATCH RFC 11/12] iommufd: vfio container FD ioctl compatibility

2022-05-10 Thread David Gibson
On Tue, May 10, 2022 at 04:00:09PM -0300, Jason Gunthorpe wrote: > On Tue, May 10, 2022 at 05:12:04PM +1000, David Gibson wrote: > > On Mon, May 09, 2022 at 11:00:41AM -0300, Jason Gunthorpe wrote: > > > On Mon, May 09, 2022 at 04:01:52PM +1000, David Gibson wrote: > >

Re: [PATCH RFC 11/12] iommufd: vfio container FD ioctl compatibility

2022-05-10 Thread David Gibson
On Mon, May 09, 2022 at 11:00:41AM -0300, Jason Gunthorpe wrote: > On Mon, May 09, 2022 at 04:01:52PM +1000, David Gibson wrote: > > > > The default iommu_domain that the iommu driver creates will be used > > > here, it is up to the iommu driver to choose something

Re: [PATCH RFC 11/12] iommufd: vfio container FD ioctl compatibility

2022-05-09 Thread David Gibson
On Fri, May 06, 2022 at 09:48:37AM -0300, Jason Gunthorpe wrote: > On Fri, May 06, 2022 at 03:25:03PM +1000, David Gibson wrote: > > On Thu, May 05, 2022 at 04:07:28PM -0300, Jason Gunthorpe wrote: > > > > When the iommu_domain is created I want to have a > > > io

Re: [PATCH RFC 11/12] iommufd: vfio container FD ioctl compatibility

2022-05-09 Thread David Gibson
On Fri, May 06, 2022 at 10:42:21AM +, Tian, Kevin wrote: > > From: David Gibson > > Sent: Friday, May 6, 2022 1:25 PM > > > > > > > > When the iommu_domain is created I want to have a > > > iommu-driver-specific struct, so PPC can cust

Re: [PATCH RFC 11/12] iommufd: vfio container FD ioctl compatibility

2022-05-05 Thread David Gibson
On Thu, May 05, 2022 at 04:07:28PM -0300, Jason Gunthorpe wrote: > On Mon, May 02, 2022 at 05:30:05PM +1000, David Gibson wrote: > > > > It is a bit more CPU work since maps in the lower range would have to > > > be copied over, but conceptually the model matches the HW n

Re: [PATCH RFC 11/12] iommufd: vfio container FD ioctl compatibility

2022-05-02 Thread David Gibson
On Fri, Apr 29, 2022 at 09:48:38AM -0300, Jason Gunthorpe wrote: > On Fri, Apr 29, 2022 at 04:20:36PM +1000, David Gibson wrote: > > > > I think PPC and S390 are solving the same problem here. I think S390 > > > is going to go to a SW nested model where it has an iommu

Re: [PATCH RFC 11/12] iommufd: vfio container FD ioctl compatibility

2022-05-02 Thread David Gibson
On Fri, Apr 29, 2022 at 09:50:30AM -0300, Jason Gunthorpe wrote: > On Fri, Apr 29, 2022 at 04:22:56PM +1000, David Gibson wrote: > > On Fri, Apr 29, 2022 at 01:21:30AM +, Tian, Kevin wrote: > > > > From: Jason Gunthorpe > > > > Sent:

Re: [PATCH RFC 08/12] iommufd: IOCTLs for the io_pagetable

2022-04-30 Thread David Gibson
On Fri, Apr 29, 2022 at 09:54:42AM -0300, Jason Gunthorpe wrote: > On Fri, Apr 29, 2022 at 04:00:14PM +1000, David Gibson wrote: > > > But I don't have a use case in mind? The simplified things I know > > > about want to attach their devices then allocate valid IOVA, they &g

Re: [PATCH RFC 11/12] iommufd: vfio container FD ioctl compatibility

2022-04-29 Thread David Gibson
On Thu, Apr 28, 2022 at 12:10:37PM -0300, Jason Gunthorpe wrote: > On Fri, Apr 29, 2022 at 12:53:16AM +1000, David Gibson wrote: > > > 2) Costly GUPs. pseries (the most common ppc machine type) always > > expects a (v)IOMMU. That means that unlike the common x86 model of a &

Re: [PATCH RFC 11/12] iommufd: vfio container FD ioctl compatibility

2022-04-29 Thread David Gibson
t; domain then the existing aperture should still work? Maybe. There might be several different ways to represent it, but the vital piece is that any individual device (well, group, technically) must atomically join/leave both windows at once. -- David Gibson| I'l

Re: [PATCH RFC 08/12] iommufd: IOCTLs for the io_pagetable

2022-04-29 Thread David Gibson
On Thu, Apr 28, 2022 at 11:22:58AM -0300, Jason Gunthorpe wrote: > On Thu, Apr 28, 2022 at 03:58:30PM +1000, David Gibson wrote: > > On Thu, Mar 31, 2022 at 09:58:41AM -0300, Jason Gunthorpe wrote: > > > On Thu, Mar 31, 2022 at 03:36:29PM +1100,

Re: [PATCH RFC 11/12] iommufd: vfio container FD ioctl compatibility

2022-04-28 Thread David Gibson
y failing this outright if there's anything other than exactly one IOMMU group bound to the container / IOAS (which I think might be what VFIO itself does now). Handling that with a device centric API gets somewhat fiddlier, of course. -- David Gibson

Re: [PATCH RFC 08/12] iommufd: IOCTLs for the io_pagetable

2022-04-28 Thread David Gibson
On Thu, Mar 31, 2022 at 09:58:41AM -0300, Jason Gunthorpe wrote: > On Thu, Mar 31, 2022 at 03:36:29PM +1100, David Gibson wrote: > > > > +/** > > > + * struct iommu_ioas_iova_ranges - ioctl(IOMMU_IOAS_IOVA_RANGES) > > > + * @size: sizeof(struct iommu_ioas_iova_ra

Re: [PATCH RFC 08/12] iommufd: IOCTLs for the io_pagetable

2022-03-30 Thread David Gibson
ID to change the mapping of > + * @iova: IOVA to start the unmapping at > + * @length: Number of bytes to unmap > + * > + * Unmap an IOVA range. The iova/length must exactly match a range > + * used with IOMMU_IOAS_PAGETABLE_MAP, or be the values 0 & U6

Re: [RFC 20/20] Doc: Add documentation for /dev/iommu

2021-10-29 Thread David Gibson
; +framework should call iommufd_bind_device(). When the device fd is > +closed by the user, iommufd_unbind_device() should be called > +automatically:: > + > + struct iommufd_device * > + iommufd_bind_device(int fd, struct device *dev, > +u64 dev_cookie);

Re: [RFC 13/20] iommu: Extend iommu_at[de]tach_device() for multiple devices group

2021-10-26 Thread David Gibson
On Mon, Oct 25, 2021 at 08:36:02PM -0300, Jason Gunthorpe wrote: > On Tue, Oct 26, 2021 at 12:16:43AM +1100, David Gibson wrote: > > If you attach devices A and B (both in group X) to IOAS 1, then detach > > device A, what happens? Do you detach both devices? Or do you have a >

Re: [RFC 13/20] iommu: Extend iommu_at[de]tach_device() for multiple devices group

2021-10-25 Thread David Gibson
On Mon, Oct 25, 2021 at 09:14:10AM -0300, Jason Gunthorpe wrote: > On Mon, Oct 25, 2021 at 04:14:56PM +1100, David Gibson wrote: > > On Mon, Oct 18, 2021 at 01:32:38PM -0300, Jason Gunthorpe wrote: > > > On Mon, Oct 18, 2021 at 02:57:12PM +1100, David Gibson wrote: > > &

Re: [RFC 13/20] iommu: Extend iommu_at[de]tach_device() for multiple devices group

2021-10-24 Thread David Gibson
On Mon, Oct 18, 2021 at 01:32:38PM -0300, Jason Gunthorpe wrote: > On Mon, Oct 18, 2021 at 02:57:12PM +1100, David Gibson wrote: > > > The first user might read this. Subsequent users are likely to just > > copy paste examples from earlier things without fully understanding >

Re: [RFC 11/20] iommu/iommufd: Add IOMMU_IOASID_ALLOC/FREE

2021-10-24 Thread David Gibson
On Thu, Oct 14, 2021 at 06:53:01AM +, Tian, Kevin wrote: > > From: David Gibson > > Sent: Thursday, October 14, 2021 1:00 PM > > > > On Wed, Oct 13, 2021 at 07:00:58AM +, Tian, Kevin wrote: > > > > From: David Gibson > > > > Sent: Friday

Re: [RFC 11/20] iommu/iommufd: Add IOMMU_IOASID_ALLOC/FREE

2021-10-17 Thread David Gibson
On Thu, Oct 14, 2021 at 11:52:08AM -0300, Jason Gunthorpe wrote: > On Thu, Oct 14, 2021 at 03:53:33PM +1100, David Gibson wrote: > > > > My feeling is that qemu should be dealing with the host != target > > > case, not the kernel. > > > > > > The ker

Re: [RFC 13/20] iommu: Extend iommu_at[de]tach_device() for multiple devices group

2021-10-17 Thread David Gibson
On Thu, Oct 14, 2021 at 07:06:07AM +, Tian, Kevin wrote: > > From: David Gibson > > Sent: Thursday, October 14, 2021 1:24 PM > > > > On Sun, Sep 19, 2021 at 02:38:41PM +0800, Liu Yi L wrote: > > > From: Lu Baolu > > > > > > Thes

Re: [RFC 11/20] iommu/iommufd: Add IOMMU_IOASID_ALLOC/FREE

2021-10-14 Thread David Gibson
On Wed, Oct 13, 2021 at 07:00:58AM +, Tian, Kevin wrote: > > From: David Gibson > > Sent: Friday, October 1, 2021 2:11 PM > > > > On Sun, Sep 19, 2021 at 02:38:39PM +0800, Liu Yi L wrote: > > > This patch adds IOASID allocation/free interface per iommufd

Re: [RFC 11/20] iommu/iommufd: Add IOMMU_IOASID_ALLOC/FREE

2021-10-14 Thread David Gibson
On Mon, Oct 11, 2021 at 03:49:14PM -0300, Jason Gunthorpe wrote: > On Mon, Oct 11, 2021 at 05:02:01PM +1100, David Gibson wrote: > > > > This means we cannot define an input that has a magic HW specific > > > value. > > > > I'm not entirely sure what you m

Re: [RFC 11/20] iommu/iommufd: Add IOMMU_IOASID_ALLOC/FREE

2021-10-14 Thread David Gibson
On Mon, Oct 11, 2021 at 09:49:57AM +0100, Jean-Philippe Brucker wrote: > On Mon, Oct 11, 2021 at 05:02:01PM +1100, David Gibson wrote: > > qemu wants to emulate a PAPR vIOMMU, so it says (via interfaces yet to > > be determined) that it needs an IOAS where things can be mapped in

Re: [RFC 13/20] iommu: Extend iommu_at[de]tach_device() for multiple devices group

2021-10-14 Thread David Gibson
; > goto out_unlock; > } > @@ -3368,6 +3384,7 @@ static int iommu_group_init_user_dma(struct iommu_group > *group, > > group->user_dma_owner_id = owner; > refcount_set(>owner_cnt, 1); > + refcount_set(>attach_cnt, 0); >

Re: [RFC 11/20] iommu/iommufd: Add IOMMU_IOASID_ALLOC/FREE

2021-10-11 Thread David Gibson
On Fri, Oct 01, 2021 at 09:22:25AM -0300, Jason Gunthorpe wrote: > On Fri, Oct 01, 2021 at 04:13:58PM +1000, David Gibson wrote: > > On Tue, Sep 21, 2021 at 02:44:38PM -0300, Jason Gunthorpe wrote: > > > On Sun, Sep 19, 2021 at 02:38:39PM +0800, Liu Yi L wrote: > > &g

Re: [RFC 07/20] iommu/iommufd: Add iommufd_[un]bind_device()

2021-10-10 Thread David Gibson
On Thu, Oct 07, 2021 at 08:35:03AM -0300, Jason Gunthorpe wrote: > On Thu, Oct 07, 2021 at 12:23:13PM +1100, David Gibson wrote: > > On Fri, Oct 01, 2021 at 09:43:22AM -0300, Jason Gunthorpe wrote: > > > On Thu, Sep 30, 2021 at 01:10:29PM +1000, David Gibson wrote: > >

Re: [RFC 07/20] iommu/iommufd: Add iommufd_[un]bind_device()

2021-10-06 Thread David Gibson
On Fri, Oct 01, 2021 at 09:43:22AM -0300, Jason Gunthorpe wrote: > On Thu, Sep 30, 2021 at 01:10:29PM +1000, David Gibson wrote: > > On Wed, Sep 29, 2021 at 09:24:57AM -0300, Jason Gunthorpe wrote: > > > On Wed, Sep 29, 2021 at 03:25:54PM +1000, David Gibson wrote: > &

Re: [RFC 11/20] iommu/iommufd: Add IOMMU_IOASID_ALLOC/FREE

2021-10-01 Thread David Gibson
> actual geometry including any holes via a query. So part of the point of specifying start/end addresses is that explicitly querying holes shouldn't be necessary: if the requested range crosses a hole, it should fail. If you didn't really need all that range, you shouldn't have asked for it. Which me

Re: [RFC 11/20] iommu/iommufd: Add IOMMU_IOASID_ALLOC/FREE

2021-10-01 Thread David Gibson
; + * User should call IOMMU_CHECK_EXTENSION for future extensions. > + * > + * @argsz: user filled size of this data. > + * @flags: additional information for IOASID allocation. > + * @type:I/O address space page table type. > + * @addr_width:address wi

Re: [RFC 06/20] iommu: Add iommu_device_init[exit]_user_dma interfaces

2021-09-30 Thread David Gibson
On Thu, Sep 30, 2021 at 07:28:18PM -0300, Jason Gunthorpe wrote: > On Thu, Sep 30, 2021 at 01:09:22PM +1000, David Gibson wrote: > > > > The *admin* the one responsible to understand the groups, not the > > > applications. The admin has no idea what a group FD is - they

Re: [RFC 06/20] iommu: Add iommu_device_init[exit]_user_dma interfaces

2021-09-29 Thread David Gibson
On Wed, Sep 29, 2021 at 07:31:08AM +, Tian, Kevin wrote: > > From: David Gibson > > Sent: Wednesday, September 29, 2021 2:35 PM > > > > On Wed, Sep 29, 2021 at 05:38:56AM +, Tian, Kevin wrote: > > > > From: David Gibson > > > &

Re: [RFC 06/20] iommu: Add iommu_device_init[exit]_user_dma interfaces

2021-09-29 Thread David Gibson
On Wed, Sep 29, 2021 at 09:57:16AM -0300, Jason Gunthorpe wrote: > On Wed, Sep 29, 2021 at 04:35:19PM +1000, David Gibson wrote: > > > Yes, exactly. And with a group interface it's obvious it has to > > understand it. With the non-group interface, you can get to this >

Re: [RFC 07/20] iommu/iommufd: Add iommufd_[un]bind_device()

2021-09-29 Thread David Gibson
On Wed, Sep 29, 2021 at 09:24:57AM -0300, Jason Gunthorpe wrote: 65;6402;1c> On Wed, Sep 29, 2021 at 03:25:54PM +1000, David Gibson wrote: > > > > +struct iommufd_device { > > > + unsigned int id; > > > + struct iommufd_ctx *ictx; > > > + struct dev

Re: [RFC 08/20] vfio/pci: Add VFIO_DEVICE_BIND_IOMMUFD

2021-09-29 Thread David Gibson
On Wed, Sep 29, 2021 at 06:41:00AM +, Tian, Kevin wrote: > > From: David Gibson > > Sent: Wednesday, September 29, 2021 2:01 PM > > > > On Sun, Sep 19, 2021 at 02:38:36PM +0800, Liu Yi L wrote: > > > This patch adds VFIO_DEVICE_BIND_IOMMUFD for userspace

Re: [RFC 02/20] vfio: Add device class for /dev/vfio/devices

2021-09-29 Thread David Gibson
On Wed, Sep 29, 2021 at 01:05:21PM -0600, Alex Williamson wrote: > On Wed, 29 Sep 2021 12:08:59 +1000 > David Gibson wrote: > > > On Sun, Sep 19, 2021 at 02:38:30PM +0800, Liu Yi L wrote: > > > This patch introduces a new interface (/dev/vfio/devices/$DEVICE) for > >

Re: [RFC 06/20] iommu: Add iommu_device_init[exit]_user_dma interfaces

2021-09-29 Thread David Gibson
On Wed, Sep 29, 2021 at 05:38:56AM +, Tian, Kevin wrote: > > From: David Gibson > > Sent: Wednesday, September 29, 2021 12:56 PM > > > > > > > > Unlike vfio, iommufd adopts a device-centric design with all group > > > logistics hidden behi

Re: [RFC 07/20] iommu/iommufd: Add iommufd_[un]bind_device()

2021-09-29 Thread David Gibson
fd.h > new file mode 100644 > index ..1603a13937e9 > --- /dev/null > +++ b/include/linux/iommufd.h > @@ -0,0 +1,38 @@ > +/* SPDX-License-Identifier: GPL-2.0-only */ > +/* > + * IOMMUFD API definition > + * > + * Copyright (C) 2021 Intel Corporation > +

Re: [RFC 08/20] vfio/pci: Add VFIO_DEVICE_BIND_IOMMUFD

2021-09-29 Thread David Gibson
d > + * > + * The user should provide a device cookie when calling this ioctl. The > + * cookie is later used in iommufd for capability query, iotlb invalidation > + * and I/O fault handling. > + * > + * User is not allowed to access the device before the binding operation > + * is c

Re: [RFC 10/20] iommu/iommufd: Add IOMMU_DEVICE_GET_INFO

2021-09-29 Thread David Gibson
u64 dev_cookie; > + __u64 pgsize_bitmap; > + __u32 addr_width; I think this is where you should be reporting available IOVA windows, rather than just an address width. I know that for ppc a real situation will be to have two different windows of different sizes: that is the effective address widt

Re: [RFC 06/20] iommu: Add iommu_device_init[exit]_user_dma interfaces

2021-09-28 Thread David Gibson
by userspace if all devices are in > + * one of the following states: > + * - driver-less > + * - bound to an allowed driver > + * - a PCI interconnect device > + */ > +static int device_user_dma_viable(struct device *dev, void *data) I think this wants a "les

Re: [RFC 03/20] vfio: Add vfio_[un]register_device()

2021-09-28 Thread David Gibson
;); > @@ -2542,6 +2654,7 @@ static int __init vfio_init(void) > static void __exit vfio_cleanup(void) > { > WARN_ON(!list_empty(_list)); > + WARN_ON(!list_empty(_list)); > > #ifdef CONFIG_VFIO_NOIOMMU > vfio_unregister_iommu_driver(_noiommu_ops); > dif

Re: [RFC 02/20] vfio: Add device class for /dev/vfio/devices

2021-09-28 Thread David Gibson
> +static char *vfio_device_devnode(struct device *dev, umode_t *mode) > +{ > + return kasprintf(GFP_KERNEL, "vfio/devices/%s", dev_name(dev)); Others have pointed out some problems with the use of dev_name() here. I'll add that I think you'll make things much easier if

Re: [RFC 04/20] iommu: Add iommu_device_get_info interface

2021-09-28 Thread David Gibson
_devattr attr, void > *data); > + > #else /* CONFIG_IOMMU_API */ > > struct iommu_ops {}; > @@ -999,6 +1012,12 @@ static inline struct iommu_fwspec > *dev_iommu_fwspec_get(struct device *dev) > { > return NULL; > } > + > +static inline int iommu_device_get_info

Re: [RFC v2] /dev/iommu uAPI proposal

2021-08-10 Thread David Gibson
On Fri, Aug 06, 2021 at 09:32:11AM -0300, Jason Gunthorpe wrote: > On Fri, Aug 06, 2021 at 02:45:26PM +1000, David Gibson wrote: > > > Well, that's kind of what I'm doing. PCI currently has the notion of > > "default" address space for a RID, but there's no gu

Re: [RFC v2] /dev/iommu uAPI proposal

2021-08-09 Thread David Gibson
On Mon, Aug 09, 2021 at 08:34:06AM +, Tian, Kevin wrote: > > From: David Gibson > > Sent: Friday, August 6, 2021 12:45 PM > > > > > In concept I feel the purpose of DMA endpoint is equivalent to the > > routing > > > > > info in this proposal.

Re: [RFC v2] /dev/iommu uAPI proposal

2021-08-05 Thread David Gibson
On Tue, Aug 03, 2021 at 03:19:26AM +, Tian, Kevin wrote: > > From: David Gibson > > Sent: Tuesday, August 3, 2021 9:51 AM > > > > On Wed, Jul 28, 2021 at 04:04:24AM +, Tian, Kevin wrote: > > > Hi, David, > > > > > > > From: Davi

Re: [RFC v2] /dev/iommu uAPI proposal

2021-08-05 Thread David Gibson
t according to its vIOMMU semantics. Then SVA capable devices would be plugged into that IOAS by using "PASID+address" type endpoints from those devices. -- David Gibson| I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank y

Re: [RFC v2] /dev/iommu uAPI proposal

2021-08-05 Thread David Gibson
On Wed, Aug 04, 2021 at 11:07:42AM -0300, Jason Gunthorpe wrote: > On Tue, Aug 03, 2021 at 11:58:54AM +1000, David Gibson wrote: > > > I'd rather deduce the endpoint from a collection of devices than the > > > other way around... > > > > Which I think is confusi

Re: [RFC v2] /dev/iommu uAPI proposal

2021-08-02 Thread David Gibson
On Fri, Jul 30, 2021 at 11:51:23AM -0300, Jason Gunthorpe wrote: > On Mon, Jul 26, 2021 at 02:50:48PM +1000, David Gibson wrote: > > > That said, I'm still finding the various ways a device can attach to > > an ioasid pretty confusing. Here are some thoughts on some extra >

Re: [RFC v2] /dev/iommu uAPI proposal

2021-08-02 Thread David Gibson
On Wed, Jul 28, 2021 at 04:04:24AM +, Tian, Kevin wrote: > Hi, David, > > > From: David Gibson > > Sent: Monday, July 26, 2021 12:51 PM > > > > On Fri, Jul 09, 2021 at 07:48:44AM +, Tian, Kevin wrote: > > > /dev/iommu provides an unified

Re: [RFC v2] /dev/iommu uAPI proposal

2021-07-25 Thread David Gibson
type: 64-bit Parent DMA address type: Root (User Virtual Address) => one-PASID endpoint for device DMA address type: 64-bit DMA identifier type: RID+PASID Driver makes mappings for a single PASID into G1. IO_MAP operations include only a 64-bit address, bec

Re: Plan for /dev/ioasid RFC v2

2021-06-23 Thread David Gibson
before it can allow user to > access a device. As long as a device in a group either in block DMA > or switch to a new address space created via /dev/iommu FD, there's > no problem to allow user accessing it. User cannot do harm to the > world outside of the group. User know

Re: Plan for /dev/ioasid RFC v2

2021-06-23 Thread David Gibson
On Thu, Jun 17, 2021 at 08:04:38PM -0300, Jason Gunthorpe wrote: > On Thu, Jun 17, 2021 at 03:02:33PM +1000, David Gibson wrote: > > > In other words, do we really have use cases where we need to identify > > different devices IDs, even though we know they're not isolated. >

Re: Plan for /dev/ioasid RFC v2

2021-06-23 Thread David Gibson
all new > sub-systems including vdpa and new vfio only support singleton > device group via /dev/iommu... The case that worries me here is if you *thought* you had 1 device groups, but then discover a hardware bug which means two things aren't as isolated as you thought they

Re: Plan for /dev/ioasid RFC v2

2021-06-23 Thread David Gibson
another device, unavoidably. It may be rare with modern hardware, but we still can't ignore the case. Which means you can't DMA block until everything in the group is managed by a vfio-like driver. -- David Gibson| I'll have my music baroque, and my code david AT gibson

Re: Plan for /dev/ioasid RFC v2

2021-06-23 Thread David Gibson
address space in the PCI topology. In a conventional or non-vIOMMU > topology, the PCI address space is equivalent to the system memory > address space. When vIOMMU gets involved, multiple devices within the > same group must exist in the same address space. A vPCIe-to-PCI bridge > ca

Re: Plan for /dev/ioasid RFC v2

2021-06-23 Thread David Gibson
On Fri, Jun 18, 2021 at 01:21:47PM +0800, Lu Baolu wrote: > Hi David, > > On 6/17/21 1:22 PM, David Gibson wrote: > > > The iommu_group can guarantee the isolation among different physical > > > devices (represented by RIDs). But when it comes to sub-devices (ex. mdev

Re: [RFC] /dev/ioasid uAPI proposal

2021-06-23 Thread David Gibson
On Wed, Jun 23, 2021 at 07:59:21AM +, Tian, Kevin wrote: > > From: David Gibson > > Sent: Thursday, June 17, 2021 11:48 AM > > > > On Tue, Jun 08, 2021 at 10:17:56AM -0300, Jason Gunthorpe wrote: > > > On Tue, Jun 08, 2021 at 12:37:04PM +1000, David Gibs

Re: [RFC] /dev/ioasid uAPI proposal

2021-06-23 Thread David Gibson
On Fri, Jun 18, 2021 at 07:03:31PM +0200, Jean-Philippe Brucker wrote: > On Thu, Jun 17, 2021 at 01:00:14PM +1000, David Gibson wrote: > > On Thu, Jun 10, 2021 at 06:37:31PM +0200, Jean-Philippe Brucker wrote: > > > On Tue, Jun 08, 2021 at 04:31:50PM +1000, David Gibson wrote: &g

Re: Plan for /dev/ioasid RFC v2

2021-06-23 Thread David Gibson
On Thu, Jun 17, 2021 at 08:10:04PM -0300, Jason Gunthorpe wrote: > On Thu, Jun 17, 2021 at 02:45:46PM +1000, David Gibson wrote: > > On Wed, Jun 09, 2021 at 09:39:19AM -0300, Jason Gunthorpe wrote: > > > On Wed, Jun 09, 2021 at 02:24:03PM +0200, Joerg Roedel wrote: > >

Re: [RFC] /dev/ioasid uAPI proposal

2021-06-23 Thread David Gibson
On Wed, Jun 23, 2021 at 08:00:49AM +, Tian, Kevin wrote: > > From: David Gibson > > Sent: Thursday, June 17, 2021 12:08 PM > > > > On Thu, Jun 03, 2021 at 08:12:27AM +, Tian, Kevin wrote: > > > > From: David Gibson > >

Re: Plan for /dev/ioasid RFC v2

2021-06-17 Thread David Gibson
a device behind a > !ACS switch if the uAPI forces all IOASID's to be linked to a group, > not a device? > > Device centric with an report that "all devices in the group must use > the same IOASID" covers all the new functionality, keep the old, and > has a better chanc

Re: Plan for /dev/ioasid RFC v2

2021-06-17 Thread David Gibson
ttaching devices within a group to different IOASIDs we effectively need to introduce two levels of "group-like" things. First the idenfication group, then the isolation group. You're using "group" for the isolation group, but then we have to somehow expose this concept of identifi

Re: Plan for /dev/ioasid RFC v2

2021-06-17 Thread David Gibson
different group from their parent device, and therefore need not be affected by whether the parent device shares a group with some other physical device. They *might* be, but that's up to the mdev driver to determine based on what it can safely isolate. > AIUI, when we attach a "RID + SSID&

Re: [RFC] /dev/ioasid uAPI proposal

2021-06-17 Thread David Gibson
On Tue, Jun 08, 2021 at 10:17:56AM -0300, Jason Gunthorpe wrote: > On Tue, Jun 08, 2021 at 12:37:04PM +1000, David Gibson wrote: > > > > The PPC/SPAPR support allows KVM to associate a vfio group to an IOMMU > > > page table so that it can handle iotlb programming from pre

Re: Plan for /dev/ioasid RFC v2

2021-06-17 Thread David Gibson
will be both fully isolated and well identified, so for the uses cases that people seem to mostly care about here we'll still have identification group == isolation group == one device. In other words, do we really have use cases where we need to identify different devices IDs,

Re: [RFC] /dev/ioasid uAPI proposal

2021-06-17 Thread David Gibson
On Thu, Jun 03, 2021 at 08:12:27AM +, Tian, Kevin wrote: > > From: David Gibson > > Sent: Wednesday, June 2, 2021 2:15 PM > > > [...] > > > > > > > /* > > > * Get information about an I/O address space > > > * > > &g

Re: [RFC] /dev/ioasid uAPI proposal

2021-06-17 Thread David Gibson
On Tue, Jun 08, 2021 at 04:04:06PM -0300, Jason Gunthorpe wrote: > On Tue, Jun 08, 2021 at 10:53:02AM +1000, David Gibson wrote: > > On Thu, Jun 03, 2021 at 08:52:24AM -0300, Jason Gunthorpe wrote: > > > On Thu, Jun 03, 2021 at 03:13:44PM +1000, David Gibson wrote: > > &

Re: [RFC] /dev/ioasid uAPI proposal

2021-06-17 Thread David Gibson
On Thu, Jun 10, 2021 at 06:37:31PM +0200, Jean-Philippe Brucker wrote: > On Tue, Jun 08, 2021 at 04:31:50PM +1000, David Gibson wrote: > > For the qemu case, I would imagine a two stage fallback: > > > > 1) Ask for the exact IOMMU capabilities (including pageta

Re: [RFC] /dev/ioasid uAPI proposal

2021-06-08 Thread David Gibson
On Thu, Jun 03, 2021 at 09:11:05AM -0300, Jason Gunthorpe wrote: > On Thu, Jun 03, 2021 at 03:45:09PM +1000, David Gibson wrote: > > On Wed, Jun 02, 2021 at 01:58:38PM -0300, Jason Gunthorpe wrote: > > > On Wed, Jun 02, 2021 at 04:48:35PM +1000, David Gibson wrote: > > >

Re: [RFC] /dev/ioasid uAPI proposal

2021-06-08 Thread David Gibson
On Thu, Jun 03, 2021 at 07:17:23AM +, Tian, Kevin wrote: > > From: David Gibson > > Sent: Wednesday, June 2, 2021 2:15 PM > > > [...] > > > An I/O address space takes effect in the IOMMU only after it is attached > > > to a device. The device i

Re: [RFC] /dev/ioasid uAPI proposal

2021-06-08 Thread David Gibson
On Fri, Jun 04, 2021 at 12:24:08PM +0200, Jean-Philippe Brucker wrote: > On Thu, Jun 03, 2021 at 03:45:09PM +1000, David Gibson wrote: > > > > But it would certainly be possible for a system to have two > > > > different host bridges with two different IOMMUs with

Re: [RFC] /dev/ioasid uAPI proposal

2021-06-08 Thread David Gibson
On Thu, Jun 03, 2021 at 09:28:32AM -0300, Jason Gunthorpe wrote: > On Thu, Jun 03, 2021 at 03:23:17PM +1000, David Gibson wrote: > > On Wed, Jun 02, 2021 at 01:37:53PM -0300, Jason Gunthorpe wrote: > > > On Wed, Jun 02, 2021 at 04:57:52PM +1000, David Gibson wrote: > >

Re: [RFC] /dev/ioasid uAPI proposal

2021-06-07 Thread David Gibson
n PPC is done through hypercalls, so KVM needs > to know how to handle those for in-kernel acceleration. Thanks, For PAPR guests, which is the common case, yes. Bare metal POWER hosts have their own page table format. And probably some of the newer embedded ppc models have some different IOMM

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-06-07 Thread David Gibson
On Tue, Jun 01, 2021 at 09:57:12AM -0300, Jason Gunthorpe wrote: > On Tue, Jun 01, 2021 at 02:03:33PM +1000, David Gibson wrote: > > On Thu, May 27, 2021 at 03:48:47PM -0300, Jason Gunthorpe wrote: > > > On Thu, May 27, 2021 at 02:58:30PM +1000, David Gibson wrote: > >

Re: [RFC] /dev/ioasid uAPI proposal

2021-06-07 Thread David Gibson
On Thu, Jun 03, 2021 at 06:49:20AM +, Tian, Kevin wrote: > > From: David Gibson > > Sent: Thursday, June 3, 2021 1:09 PM > [...] > > > > In this way the SW mode is the same as a HW mode with an infinite > > > > cache. > > > > > > &g

Re: [RFC] /dev/ioasid uAPI proposal

2021-06-07 Thread David Gibson
/dev/ioasid, > > thus increasing the complexity in container setup (or orchestration). > > Containers already needed to do this today. Container orchestration is > hard. Right to use VFIO a container already needs both /dev/vfio and one or more /dev/vfio/NNN group dev

Re: [RFC] /dev/ioasid uAPI proposal

2021-06-07 Thread David Gibson
On Thu, Jun 03, 2021 at 08:52:24AM -0300, Jason Gunthorpe wrote: > On Thu, Jun 03, 2021 at 03:13:44PM +1000, David Gibson wrote: > > > > We can still consider it a single "address space" from the IOMMU > > > perspective. What has happened is that the address tab

Re: [RFC] /dev/ioasid uAPI proposal

2021-06-03 Thread David Gibson
d to support 2), Isn't (2) the equivalent of using the using the host-managed pagetable then doing a giant MAP of all your user address space into it? But maybe we should identify that case explicitly in case the host can optimize it. > 3) and beyond? If so, it seems that we need some in

Re: [RFC] /dev/ioasid uAPI proposal

2021-06-03 Thread David Gibson
On Wed, Jun 02, 2021 at 02:19:30PM -0300, Jason Gunthorpe wrote: > On Wed, Jun 02, 2021 at 04:15:07PM +1000, David Gibson wrote: > > > Is there a compelling reason to have all the IOASIDs handled by one > > FD? > > There was an answer on this, if every PASID needs an IOA

Re: [RFC] /dev/ioasid uAPI proposal

2021-06-03 Thread David Gibson
On Wed, Jun 02, 2021 at 01:16:48PM -0300, Jason Gunthorpe wrote: > On Wed, Jun 02, 2021 at 04:32:27PM +1000, David Gibson wrote: > > > I agree with Jean-Philippe - at the very least erasing this > > > information needs a major rational - but I don't really see why it > &g

Re: [RFC] /dev/ioasid uAPI proposal

2021-06-03 Thread David Gibson
On Wed, Jun 02, 2021 at 01:58:38PM -0300, Jason Gunthorpe wrote: > On Wed, Jun 02, 2021 at 04:48:35PM +1000, David Gibson wrote: > > > > /* Bind guest I/O page table */ > > > > bind_data = { > > > > .ioasid = gv

Re: [RFC] /dev/ioasid uAPI proposal

2021-06-03 Thread David Gibson
On Thu, Jun 03, 2021 at 02:49:56AM +, Tian, Kevin wrote: > > From: Jason Gunthorpe > > Sent: Thursday, June 3, 2021 12:59 AM > > > > On Wed, Jun 02, 2021 at 04:48:35PM +1000, David Gibson wrote: > > > > > /* Bind guest I/O p

Re: [RFC] /dev/ioasid uAPI proposal

2021-06-03 Thread David Gibson
On Wed, Jun 02, 2021 at 01:37:53PM -0300, Jason Gunthorpe wrote: > On Wed, Jun 02, 2021 at 04:57:52PM +1000, David Gibson wrote: > > > I don't think presence or absence of a group fd makes a lot of > > difference to this design. Having a group fd just means we attach > &

Re: [RFC] /dev/ioasid uAPI proposal

2021-06-03 Thread David Gibson
table > > is changed and invalidation is not issued then then the implementation > > is free to ignore the changes. > > > > In this way the SW mode is the same as a HW mode with an infinite > > cache. > > > > The collaposed shadow page table is reall

Re: [RFC] /dev/ioasid uAPI proposal

2021-06-02 Thread David Gibson
le vfio devices can be bound to a single ioasid_fd, but a single > >* vfio device should not be bound to multiple ioasid_fd's. > >* > >* Input parameters: > >* - ioasid_fd; > >* > >* Return: 0 on success, -errno on failur

Re: [RFC] /dev/ioasid uAPI proposal

2021-06-02 Thread David Gibson
on there? > > Most likely passing the /dev/ioasid into KVM's FD (or vicevera) is the > right answer. Eg if ARM needs to get the VMID from KVM and set it to > ioasid then a KVM "ioctl set_arm_vmid(/dev/ioasid)" call is > reasonable. Certainly better than the symbol get su

Re: [RFC] /dev/ioasid uAPI proposal

2021-06-02 Thread David Gibson
On Thu, May 27, 2021 at 07:58:12AM +, Tian, Kevin wrote: > /dev/ioasid provides an unified interface for managing I/O page tables for > devices assigned to userspace. Device passthrough frameworks (VFIO, vDPA, > etc.) are expected to use this interface instead of creating their own logic >

Re: [RFC] /dev/ioasid uAPI proposal

2021-06-02 Thread David Gibson
ce_fd1, VFIO_ATTACH_IOASID, _data); > > > > /* Bind guest I/O page table. Because SET_PASID_TABLE has been > > * used, the kernel will not update the PASID table. Instead, just > > * track the bound I/O page table for handling invalidation and > >

Re: [RFC] /dev/ioasid uAPI proposal

2021-06-02 Thread David Gibson
ze these somehow. > > But that is too complicated and far out for me at least to guess on at > this point.. > > > > Still a little unsure why the vPASID is here not on the gva_ioasid. Is > > > there any scenario where we want different vpasid's for the same > > &g

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-06-01 Thread David Gibson
On Thu, May 27, 2021 at 11:55:00PM +0530, Kirti Wankhede wrote: > > > On 5/27/2021 10:30 AM, David Gibson wrote: > > On Wed, May 26, 2021 at 02:48:03AM +0530, Kirti Wankhede wrote: > > > > > > > > > On 5/26/2021 1:22 AM, Jason Gunthorpe wrote: > &

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-06-01 Thread David Gibson
On Thu, May 27, 2021 at 04:06:20PM -0300, Jason Gunthorpe wrote: > On Thu, May 27, 2021 at 02:53:42PM +1000, David Gibson wrote: > > > > > If the physical device had a bug which meant the mdevs *weren't* > > > > properly isolated from each other, then those mde

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-06-01 Thread David Gibson
On Thu, May 27, 2021 at 03:48:47PM -0300, Jason Gunthorpe wrote: > On Thu, May 27, 2021 at 02:58:30PM +1000, David Gibson wrote: > > On Tue, May 25, 2021 at 04:52:57PM -0300, Jason Gunthorpe wrote: > > > On Wed, May 26, 2021 at 12:56:30AM +0530, Kirti Wankhede wrote: > > &g

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-05-27 Thread David Gibson
On Mon, May 24, 2021 at 08:37:44PM -0300, Jason Gunthorpe wrote: > On Mon, May 24, 2021 at 05:52:58PM +1000, David Gibson wrote: > > > > > I don't really see a semantic distinction between "always one-device > > > > groups" and "groups don't

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-05-27 Thread David Gibson
erly by fully linking the struct > pci_device of the VF throughout the entire VFIO scheme, including the > group and container, while still allowing override of various VFIO > operations. > > Jason > -- David Gibson| I'll have my music baroque, and my code da

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-05-27 Thread David Gibson
> > pci_device of the VF throughout the entire VFIO scheme, including the > > group and container, while still allowing override of various VFIO > > operations. > > > > Jason > > > -- David Gibson| I'll have my music baroque, and my co

Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

2021-05-24 Thread David Gibson
On Thu, May 13, 2021 at 10:59:38AM -0300, Jason Gunthorpe wrote: > On Thu, May 13, 2021 at 03:48:19PM +1000, David Gibson wrote: > > On Mon, May 03, 2021 at 01:15:18PM -0300, Jason Gunthorpe wrote: > > > On Thu, Apr 29, 2021 at 01:04:05PM +1000, David Gibson wrote: > >

  1   2   >