On 25.09.25 12:51, Thomas Hellström wrote: >>> In that case I strongly suggest to add a private DMA-buf interface >>> for the DMA- >>> bufs exported by vfio-pci which returns which BAR and offset the >>> DMA-buf >>> represents. > > @Christian, Is what you're referring to here the "dma_buf private > interconnect" we've been discussing previously, now only between vfio- > pci and any interested importers instead of private to a known exporter > and importer? > > If so I have a POC I can post as an RFC on a way to negotiate such an > interconnect.
I was just about to write something up as well, but feel free to go ahead if you already have something. >> Does this private dmabuf interface already exist or does it need to >> be created >> from the ground up? Every driver which supports both exporting and importing DMA-buf has code to detect when somebody tries to re-import a buffer previously exported from the same device. Now some drivers like amdgpu and I think XE as well also detect if the buffer is from another device handled by the same driver which potentially have private interconnects (XGMI or similar). See function amdgpu_dmabuf_is_xgmi_accessible() in amdgpu_dma_buf.c for an example. >> If it already exists, could you please share an example/reference of >> how you >> have used it with amdgpu or other drivers? Well what's new is that we need to do this between two drivers unreleated to each other. As far as I know previously that was all inside AMD drivers for example, while in this case vfio is a common vendor agnostic driver. So we should probably make sure to get that right and vendor agnostic etc.... >> If it doesn't exist, I was wondering if it should be based on any >> particular best >> practices/ideas (or design patterns) that already exist in other >> drivers? > > @Vivek, another question: Also on the guest side we're exporting dma- > mapped adresses that are imported and somehow decoded by the guest > virtio-gpu driver? Is something similar needed there? > > Also how would the guest side VF driver know that what is assumed to be > a PF on the same device is actually a PF on the same device and not a > completely different device with another driver? (In which case I > assume it would like to export a system dma-buf)? Another question is how is lifetime handled? E.g. does the guest know that a DMA-buf exists for it's BAR area? Regards, Christian. > > Thanks, > Thomas > > > >> >>> >>> Ideally using the same structure Qemu used to provide the offset to >>> the vfio- >>> pci driver, but not a must have. >>> >>> This way the driver for the GPU PF (XE) can leverage this >>> interface, validates >>> that the DMA-buf comes from a VF it feels responsible for and do >>> the math to >>> figure out in which parts of the VRAM needs to be accessed to >>> scanout the >>> picture. >> Sounds good. This is definitely a viable path forward and it looks >> like we are all >> in agreement with this idea. >> >> I guess we can start exploring how to implement the private dmabuf >> interface >> mechanism right away. >> >> Thanks, >> Vivek >> >>> >>> This way this private vfio-pci interface can also be used by >>> iommufd for >>> example. >>> >>> Regards, >>> Christian. >>> >>>> >>>> Thanks, >>>> Vivek >>>> >>>>> >>>>> Regards, >>>>> Christian. >>>>> >>>>>> >>>>>>> What Simona agreed on is exactly what I proposed as well, >>>>>>> that you >>>>>>> get a private interface for exactly that use case. >>>>>> >>>>>> A "private" interface to exchange phys_addr_t between at >>>>>> least >>>>>> VFIO/KVM/iommufd - sure no complaint with that. >>>>>> >>>>>> Jason >>>> >> >
