Extend the vfio-user protocol specification to allow servers to pass multiple file descriptors for different sparse memory areas within a single device region with capability VFIO_REGION_INFO_CAP_SPARSE_MMAP_FDS.
VFIO_REGION_INFO_CAP_SPARSE_MMAP_FDS defines an array of sparse mmap areas where each area contains: - offset: Start of the area within the device region (BAR). - fd_offset: Start of the mapping within the backing file descriptor. - size: Size of the mappable area. - fd_index: Zero-based index into the array of file descriptors received in the SCM_RIGHTS ancillary data of the VFIO_USER_DEVICE_GET_REGION_INFO reply message. Signed-off-by: Naman Gulati <[email protected]> --- docs/interop/vfio-user.rst | 71 +++++++++++++++++++++++++++++++++++--- 1 file changed, 67 insertions(+), 4 deletions(-) diff --git a/docs/interop/vfio-user.rst b/docs/interop/vfio-user.rst index 1c92f442d2..5744f58e31 100644 --- a/docs/interop/vfio-user.rst +++ b/docs/interop/vfio-user.rst @@ -125,7 +125,10 @@ Sparse Regions A region can be memory-mappable in whole or in part. When only a subset of a region can be mapped by the client, a ``VFIO_REGION_INFO_CAP_SPARSE_MMAP`` capability is included in the region info reply. This capability describes -which portions can be mapped by the client. +which portions can be mapped by the client. When sparse mmap areas are backed +by multiple distinct file descriptors, a +``VFIO_REGION_INFO_CAP_SPARSE_MMAP_FDS`` capability is included in the region +info reply instead. .. Note:: For example, in a virtual NVMe controller, sparse regions can be used so @@ -859,9 +862,9 @@ Reply * ``VFIO_REGION_INFO_FLAG_READ`` allows client read access to the region. * ``VFIO_REGION_INFO_FLAG_WRITE`` allows client write access to the region. * ``VFIO_REGION_INFO_FLAG_MMAP`` specifies the client can mmap() the region. - When this flag is set, the reply will include a file descriptor in its - meta-data. On ``AF_UNIX`` sockets, the file descriptors will be passed as - ``SCM_RIGHTS`` type ancillary data. + When this flag is set, the reply will include one or more file descriptors + in its meta-data. On ``AF_UNIX`` sockets, the file descriptors will be passed + as ``SCM_RIGHTS`` type ancillary data. * ``VFIO_REGION_INFO_FLAG_CAPS`` indicates additional capabilities found in the reply. @@ -950,6 +953,66 @@ The VFIO sparse mmap area is defined in ``<linux/vfio.h>`` (``struct vfio_region_info_cap_sparse_mmap``). +VFIO sparse mmap fds cap header +""""""""""""""""""""""""""""""" + ++----------------------+--------------------------------------+ +| Name | Value | ++======================+======================================+ +| id | VFIO_REGION_INFO_CAP_SPARSE_MMAP_FDS | ++----------------------+--------------------------------------+ +| version | 0x1 | ++----------------------+--------------------------------------+ +| next | <next> | ++----------------------+--------------------------------------+ +| sparse mmap fds info | VFIO region info sparse mmap fds | ++----------------------+--------------------------------------+ + +This capability is defined when a region supports direct access by the client +via mmap() where different sparse mmap areas are backed by distinct file +descriptors in the ``SCM_RIGHTS`` ancillary data array. Capability ID +``VFIO_REGION_INFO_CAP_SPARSE_MMAP_FDS`` is defined as 16 (0x10) to avoid +clashing with kernel VFIO capabilities. + +VFIO region info cap sparse mmap fds +"""""""""""""""""""""""""""""""""""" + ++-----------+--------+------+ +| Name | Offset | Size | ++===========+========+======+ +| nr_areas | 0 | 4 | ++-----------+--------+------+ +| reserved | 4 | 4 | ++-----------+--------+------+ +| offset | 8 | 8 | ++-----------+--------+------+ +| fd_offset | 16 | 8 | ++-----------+--------+------+ +| size | 24 | 8 | ++-----------+--------+------+ +| fd_index | 32 | 4 | ++-----------+--------+------+ +| pad | 36 | 4 | ++-----------+--------+------+ +| ... | | | ++-----------+--------+------+ + +* *nr_areas* is the number of sparse mmap areas in the region. +* *offset*, *fd_offset*, *size*, and *fd_index* describe a single area that can be + mapped by the client. There will be *nr_areas* tuples of offset, fd_offset, size, + fd_index, and pad. +* *offset* specifies the start of the area within the device region (BAR). +* *fd_offset* specifies the start of the mapping within the backing file descriptor + referenced by *fd_index*. +* *fd_index* is the zero-based index into the array of file descriptors + received in the ``SCM_RIGHTS`` ancillary data of the + ``VFIO_USER_DEVICE_GET_REGION_INFO`` reply message. + +If all sparse mmap areas are backed by a single file descriptor, the server +may instead send ``VFIO_REGION_INFO_CAP_SPARSE_MMAP`` and exactly one file +descriptor in ``SCM_RIGHTS`` for compatibility with older clients. + + ``VFIO_USER_DEVICE_GET_REGION_IO_FDS`` -------------------------------------- -- 2.55.0.860.g4b6b3295ed-goog
