This patchset is an RFC version for the ioregionfd implementation in QEMU. The kernel patches are to be posted with some fixes as a v4.
For this implementation version 3 of the posted kernel patches was user: https://lore.kernel.org/kvm/cover.1613828726.git.eafanas...@gmail.com/ The future version will include support for vfio/libvfio-user. Please refer to the design discussion here proposed by Stefan: https://lore.kernel.org/all/YXpb1f3KicZxj1oj@stefanha-x1.localdomain/T/ The vfio-user version needed some bug-fixing and it was decided to send this for multiprocess first. The ioregionfd is configured currently trough the command line and each ioregionfd represent an object. This allow for easy parsing and does not require device/remote object command line option modifications. The following command line can be used to specify ioregionfd: <snip> '-object', 'x-remote-object,id=robj1,devid=lsi0,fd='+str(remote.fileno()),\ '-object', 'ioregionfd-object,id=ioreg2,devid=lsi0,iofd='+str(iord.fileno())+',bar=1',\ '-object', 'ioregionfd-object,id=ioreg3,devid=lsi0,iofd='+str(iord.fileno())+',bar=2',\ </snip> Proxy side of ioregionfd in this version uses only one file descriptor: <snip> '-device', 'x-pci-proxy-dev,id=lsi0,fd='+str(proxy.fileno())+',ioregfd='+str(iowr.fileno()), \ </snip> This is done for RFC version and my though was that next version will be for vfio-user, so I have not dedicated much effort to this command line options. The multiprocess messaging protocol was extended to support inquiries by the proxy if device has any ioregionfds. This RFC implements inquires by proxy about the type of BAR (ioregionfd or not) and the type of it (memory/io). Currently there are few limitations in this version of ioregionfd. - one ioregionfd per bar, only full bar size is supported; - one file descriptor per device for all of its ioregionfds; - each remote device runs fd handler for all its BARs in one IOThread; - proxy supports only one fd. Some of these limitations will be dropped in the future version. This RFC is to acquire the feedback/suggestions from the community on the general approach. The quick performance test was done for the remote lsi device with ioregionfd and without for both mem BARs (1 and 2) with help of the fio tool: Random R/W: read IOPS read BW write IOPS write BW no ioregionfd 889 3559KiB/s 890 3561KiB/s ioregionfd 938 3756KiB/s 939 3757KiB/s Sequential Read and Sequential Write: Sequential read Sequential write read IOPS read BW write IOPS write BW no ioregionfd 367k 1434MiB/s 76k 297MiB/s ioregionfd 374k 1459MiB/s 77.3k 302MiB/s Please review and send your feedback. Thank you! Elena Elena Ufimtseva (8): ioregionfd: introduce a syscall and memory API multiprocess: place RemoteObject definition in a header file ioregionfd: introduce memory API functions ioregionfd: Introduce IORegionDFObject type multiprocess: prepare ioregionfds for remote device multiprocess: add MPQEMU_CMD_BAR_INFO multiprocess: add ioregionfd memory region in proxy multiprocess: handle ioregionfd commands meson.build | 15 +- qapi/qom.json | 32 ++- include/exec/memory.h | 50 +++++ include/hw/remote/ioregionfd.h | 45 ++++ include/hw/remote/machine.h | 1 + include/hw/remote/mpqemu-link.h | 2 + include/hw/remote/proxy.h | 1 + include/hw/remote/remote.h | 31 +++ include/sysemu/kvm.h | 15 ++ linux-headers/ioregionfd.h | 30 +++ linux-headers/linux/kvm.h | 25 +++ accel/kvm/kvm-all.c | 132 ++++++++++++ accel/stubs/kvm-stub.c | 1 + hw/remote/ioregionfd.c | 361 ++++++++++++++++++++++++++++++++ hw/remote/message.c | 38 ++++ hw/remote/proxy.c | 66 +++++- hw/remote/remote-obj.c | 154 ++++++++++++-- softmmu/memory.c | 207 ++++++++++++++++++ Kconfig.host | 3 + MAINTAINERS | 3 + hw/remote/Kconfig | 4 + hw/remote/meson.build | 1 + meson_options.txt | 2 + scripts/meson-buildoptions.sh | 3 + 24 files changed, 1199 insertions(+), 23 deletions(-) create mode 100644 include/hw/remote/ioregionfd.h create mode 100644 include/hw/remote/remote.h create mode 100644 linux-headers/ioregionfd.h create mode 100644 hw/remote/ioregionfd.c -- 2.25.1
