On 22.09.25 14:29, Jason Gunthorpe wrote: > On Mon, Sep 22, 2025 at 02:25:15PM +0200, Christian König wrote: >> On 22.09.25 14:20, Jason Gunthorpe wrote: >>> On Mon, Sep 22, 2025 at 01:22:49PM +0200, Christian König wrote: >>> >>>> Well what exactly is happening here? You have a PF assigned to the >>>> host and a VF passed through to a guest, correct? >>>> >>>> And now the PF (from the host side) wants to access a BAR of the VF? >>> >>> Not quite. >>> >>> It is a GPU so it has a pool of VRAM. The PF can access all VRAM and >>> the VF can access some VRAM. >>> >>> They want to get a DMABUF handle for a bit of the VF's reachable VRAM >>> that the PF can import and use through it's own funciton. >> >> Yeah, where's the problem? We do that all the time. > > Well this is the problem: > >>> The use of the VF's BAR in this series is an ugly hack. The PF never >>> actually uses the VF BAR, it just hackily converts the dma_addr_t back >>> to CPU physical and figures out where it is in the VRAM pool and then >>> uses a PF centric address for it. >> >> Oh my, please absolutely don't do that! > > Great. > >> If you have a device internal connection then you need special >> handling between your PF and VF driver. > > Generic VFIO can't do something like that, so they would need to make > a special variant VFIO driver. > >> But I still don't have the full picture of who is using what here, >> e.g. who is providing the DMA-buf handle and who is using it? > > Generic VFIO is the exporer, the xe driver is the importer.
Why in the world is the exporter the generic VFIO driver? At least on AMD GPUs when you want to have a DMA-buf for a specific part of the VFs resources then you ask the hypervisor driver managing the PF for that and not the VFIO driver. Background is that the VFIO only sees the BARs of the VF and that in turn is usually only a window giving access to a fraction of the resources assigned to the VF. In other words VF BAR is 256MiB in size while 4GiB of VRAM is assigned to the VF (for example). With that design you don't run into issues with PCIe P2P in the first place. On the other hand when you want to do PCIe P2P to the VF BAR than asking the VFIO driver does make perfect sense, but that doesn't seem to be the use case here. Regards, Christian. > > Jason
