On Fri, Dec 30, 2016 at 06:09:09PM +0800, Jason Wang wrote: > Hi all: > > As the userspace vitio driver became popular, more and more request > were received for secure DMA environemt (DMAR). So this series tries > to make DMAR works for virtio/vhost. The idea is let virtio/vhost > co-work with userspace iommu implememtation. This is done through: > > - for virtio, when platform supports IOMMU (VIRTIO_F_IOMMU_PLATFORM), > virtio will not assume address_space_memory, instead a transport > specific method were introduced for querying the dma address space > and dma helpers were used in device emulation codes. > - for vhost, implement a device IOTLB by using device IOTLB API > supported by recent kernel. With this API, vhost kernel can query > IOTLB entry for a specified iova from qemu, qemu can invalidate an > arbitrary range of iova in vhost kernel. > > The device IOTLB API is totaly architecture independent, an example > implementation was done with intel iommu by: > > - implement basic ATS (Address Translation Service) for virtio-pci, > this will make device IOTLB visible for iommu driver in guest. > - implement device IOTLB descriptor processing in intel iommu (enabled > through device-iotlb=on), and trigger the device IOTLB invalidation > in vhost through iommu notifier. > > It could be easily ported to other IOMMU or architecture even if it > doesn't support device IOTLB. (e.g just invalidate the vhost IOTLB > during IOMMU IOTLB invalidation). But this will be slow since several > devics were contending userspace IOTLB entries. > > AMD IOMMU suppot for device IOTLB is ready, but it depends on other > fixes to work correctly. > > Test was done by: > > - intel_iommu=on/strict in guest. > - vfio l2fwd in guest. > > This main use case is the programs that use fixed mapping in guest > (e.g dpdk). If 1G hugepage were used in guest, thanks to the SLLPS > support, we can get 100% TLB hit rate for l2fwd in guest. > > For the normal kernel driver which uses lots of dynamic mapping and > unmapping, we may see performance penalty, this could be optimized in > the future.
This causes make check failure with Broken pipe message. Probably an unexpected interaction with vhost-user. > TODO: > - more platforms and IOMMU support (done but block by other bugs of > AMD IOMMU) > - performance optimizations (e.g merging adjacent mappings) > - non ATS support (userspace IOTLB snooping) > - using ATSCtl to disable Device IOTLB > > Changes from V3: > - rebase to HEAD > - fill all entry of IOMMUTLBEntry before notify > > Changes from V2: > - rebase to HEAD > - avoid querying dma_as each time by using vdev->dma_as directly > - fix and improve address_space_get_iotlb_entry() > - drop patch 1 (which has been posted as an independent fix) > - fix ECAP when device-iotlb=off > - fix centos6 build > > Changes from V1: > - rebase to HEAD > - avoid calling transport specific dma as fetching method each time by > caching it in vdev > - convert to use new IOMMU notifier API > - silent checkpatch warnings and fix 32bit build > - use "device-iotlb" instead of "device_iotlb" > - rename virtio_memory_map() to vhost_memory_map() and move it to vhost.c > - use memory_region_is_iommu() instead of inventing new one > > Changes from RFC: > - rebase to HEAD > - switch to use new vhost device IOTLB API > - use the new feature bit VIRITO_F_IOMMU_PLATFORM > - finalize basic ATS implementation > - add ATSR for Root port ATS transaction > - fix the iommu notifier handling during unregistering > - use snprintf() in patch 3 > - correc the loop in address_space_get_iotlb_entry() > - small tweak on the address calculation during device iotlb > descriptor processing. > > Jason Wang (10): > virtio: convert to use DMA api > intel_iommu: name vtd address space with devfn > intel_iommu: allocate new key when creating new address space > exec: introduce address_space_get_iotlb_entry() > intel_iommu: support device iotlb descriptor > virtio-pci: address space translation service (ATS) support > acpi: add ATSR for q35 > memory: handle alias for iommu notifier > memory: handle alias in memory_region_is_iommu() > vhost_net: device IOTLB support > > exec.c | 33 ++++++ > hw/block/virtio-blk.c | 2 +- > hw/char/virtio-serial-bus.c | 3 +- > hw/i386/acpi-build.c | 9 ++ > hw/i386/intel_iommu.c | 92 +++++++++++++-- > hw/i386/intel_iommu_internal.h | 13 ++- > hw/i386/x86-iommu.c | 17 +++ > hw/pci/pcie.c | 16 +++ > hw/scsi/virtio-scsi.c | 4 +- > hw/virtio/vhost-backend.c | 99 +++++++++++++++++ > hw/virtio/vhost.c | 178 > ++++++++++++++++++++++++++---- > hw/virtio/virtio-bus.c | 8 ++ > hw/virtio/virtio-pci.c | 21 ++++ > hw/virtio/virtio-pci.h | 4 + > hw/virtio/virtio.c | 57 ++++++---- > include/exec/memory.h | 8 ++ > include/hw/acpi/acpi-defs.h | 12 ++ > include/hw/i386/x86-iommu.h | 1 + > include/hw/pci/pcie.h | 4 + > include/hw/virtio/vhost-backend.h | 13 +++ > include/hw/virtio/vhost.h | 4 + > include/hw/virtio/virtio-access.h | 31 ++++-- > include/hw/virtio/virtio-bus.h | 1 + > include/hw/virtio/virtio.h | 9 +- > include/standard-headers/linux/pci_regs.h | 1 + > memory.c | 9 ++ > net/tap.c | 1 + > 27 files changed, 582 insertions(+), 68 deletions(-) > > -- > 2.7.4