Hi Leon, Sure, I'm happy to explain the TPH side. I see you've set the dma-buf patches aside for now, so I'll just put the requirement on record rather than ask for a particular API.
On the mlx5 ODP path the mkey must exist before the first dma-buf mapping, and its ST index and PH are fixed at creation. It's the same ordering problem you have with ATS. At that point mlx5 needs the ST/PH tuple and an answer to Jason's question, whether its traffic to the buffer is routed directly (PCI_P2PDMA_MAP_BUS_ADDR) rather than through the host bridge. What mlx5 needs there is the answer, not the exporter's p2pdma_provider. Wherever the route gets computed is fine, as long as the result comes back for mlx5's own attachment. In my earlier reply to Jason I said a later attach doesn't change an existing importer's route. Christian pointed out in the patch 16 thread that it can, because the exporter only commits the access path when the mapping is created. I was wrong there. The mlx5 patch re-queries the ST/PH under dmabuf->resv after each ib_umem_dmabuf_map_pages() and fails the fault if it changed; with vfio-pci the tuple only changes if the device owner sets a new one. The route check belongs in the same two places, which I think is close to what Thomas suggested. Alistair, adding you for visibility since you're picking up the mapping-type work after LPC. mlx5 TPH would be the second user of whatever lands there. I'm happy to help: I can implement the TPH side on top of it and test on a CX-8 and a peer endpoint behind a PCIe switch, including analyzer captures of the TLPs, or I can take on a piece of the work if that's useful. Thanks, Zhiping On Wed, Sep 30, 2026 at 5:13 AM Leon Romanovsky <[email protected]> wrote: > > > > On Mon, Sep 28, 2026 at 02:47:46PM -0700, Zhiping Zhang wrote: > > On Mon, Sep 28, 2026 at 10:30 AM Jason Gunthorpe <[email protected]> wrote: > > > > > > > > > > On Thu, Sep 24, 2026 at 11:17:57PM -0700, Zhiping Zhang wrote: > > > > On Thu, Sep 24, 2026 at 4:31 PM Jason Gunthorpe <[email protected]> wrote: > > > > > > > > > > > > > > > > On Wed, Sep 23, 2026 at 11:24:56PM -0700, Zhiping Zhang wrote: > > > > > > I don't think this belongs in the importer. Every in-tree dma-buf > > > > > > caller of pci_p2pdma_distance() is the exporter, in its .attach, > > > > > > clearing attach->peer2peer -- amdgpu_dma_buf.c, xe_dma_buf.c, > > > > > > habanalabs memory.c. There is no importer-side caller. > > > > > > > > > > Right, and they shouldn't be doing that, but it still has to be > > > > > checked that the st is going directly to the peer device not the host > > > > > bridge. Not using the distance, but by the PCI_P2PDMA_MAP_BUS_ADDR > > > > > indication. > > > > > > > > > > I fear you will need some of Leon's series to make that happen. > > > > > > > > > > So probably the proposed change to dmabuf ops is far too simple. > > > > > > > > > > Jason > > > > > > > > Hi Jason, > > > > > > > > Thanks for the comments. > > > > > > > > Agreed -- the tag should not be handed out unless the routing is > > > > PCI_P2PDMA_MAP_BUS_ADDR. That can be enforced conservatively on 7.3 > > > > with two changes. I can fold both into patch 4. > > > > ``` > > > > In drivers/pci/p2pdma.c, make pci_p2pdma_map_type() callable from > > > > tristate VFIO_PCI_CORE: > > > > > > > > EXPORT_SYMBOL_GPL(pci_p2pdma_map_type); > > > > > > No, that's been rejected several times already. > > > > > > > and in drivers/vfio/pci/vfio_pci_dmabuf.c, > > > > vfio_pci_dma_buf_get_pci_tph() gains: > > > > > > > > struct dma_buf_attachment *attach; > > > > > > > > if (list_empty(&dmabuf->attachments)) > > > > return -EOPNOTSUPP; > > > > > > > > list_for_each_entry(attach, &dmabuf->attachments, node) > > > > if (pci_p2pdma_map_type(priv->provider, attach->dev) != > > > > PCI_P2PDMA_MAP_BUS_ADDR) > > > > return -EOPNOTSUPP; > > > > ``` > > > > > > Also no, this has to be done via PCI_P2PDMA_MAP_BUS_ADDR which also > > > enforces putting the determination in the right place in the code > > > flow.. > > > > > > > Two properties are worth stating explicitly: > > > > > > Ah! AI! > > > > > > Jason > > > > Thanks Jason, I see what you meant by Leon's series now: > > > > https://lore.kernel.org/all/[email protected]/ > > > > Looking at v8, it seems patches 1-4 can stay functionally unchanged, > > while patch 5 uses dma_buf_p2pdma_map_type() on its own attachment for > > both the initial TPH query and revalidation. Is that what you expect? > > > > If so, would you prefer that I wait for Leon's series and rebase the > > whole stack on it, or split the series so patches 1-4 can land first > > and patch 5 follows after Leon's series? > > I would like you to join me there and explain to Christian why the > importer needs access to the exporter's internals. > > You are already second user for the same functionally. > > Thanks > > > > > Thanks, > > Zhiping
