On 6/25/26 13:00, Duan, Zhenzhong wrote:
-----Original Message-----
From: Cédric Le Goater <[email protected]>
Subject: Re: [PULL 28/41] vfio/listener: Add missing dirty tracking in
region_del
On 6/24/26 10:02, Duan, Zhenzhong wrote:
Hi Cédric,
-----Original Message-----
From: Cédric Le Goater <[email protected]>
Subject: Re: [PULL 28/41] vfio/listener: Add missing dirty tracking in
region_del
Hello Zhenzhong,
On 1/13/26 10:36, Cédric Le Goater wrote:
From: Zhenzhong Duan <[email protected]>
If a VFIO device in guest switches from passthrough(PT) domain to block
domain, the whole memory address space is unmapped, but we passed a
NULL
iotlb entry to unmap_bitmap, then bitmap query didn't happen and we lost
dirty pages.
By constructing an iotlb entry with iova = gpa for unmap_bitmap, it can
set dirty bits correctly.
For IOMMU address space, we still send NULL iotlb because VFIO don't know
the actual mappings in guest. It's vIOMMU's responsibility to send actual
unmapping notifications, e.g., vtd_address_space_unmap_in_dirty_tracking().
Signed-off-by: Zhenzhong Duan <[email protected]>
Tested-by: Giovannio Cabiddu <[email protected]>
Reviewed-by: Yi Liu <[email protected]>
Link: https://lore.kernel.org/qemu-devel/20251218062643.624796-8-
[email protected]
Signed-off-by: Cédric Le Goater <[email protected]>
---
hw/vfio/listener.c | 22 +++++++++++++++++++++-
1 file changed, 21 insertions(+), 1 deletion(-)
diff --git a/hw/vfio/listener.c b/hw/vfio/listener.c
index
62699cb772d786c1510318dff73973ef4d297177..813621f22f8b5ec284388f9c5f71
9525ec5f282c 100644
--- a/hw/vfio/listener.c
+++ b/hw/vfio/listener.c
@@ -713,14 +713,34 @@ static void
vfio_listener_region_del(MemoryListener
*listener,
if (try_unmap) {
bool unmap_all = false;
+ IOMMUTLBEntry entry = {}, *iotlb = NULL;
if (int128_eq(llsize, int128_2_64())) {
assert(!iova);
unmap_all = true;
llsize = int128_zero();
}
+
+ /*
+ * Fake an IOTLB entry for identity mapping which is needed by dirty
+ * tracking when switch out of PT domain. In fact, in unmap_bitmap,
+ * only translated_addr field is used to set dirty bitmap.
+ *
+ * Note: When switch into PT domain from DMA domain, the whole
IOMMU
+ * MR is deleted without iotlb, before that happen, we depend on
+ * vIOMMU to send unmap notification with accurate iotlb entry to
+ * VFIO. See vtd_address_space_unmap_in_dirty_tracking() for example,
+ * it is triggered during switching to block domain because vtd does
+ * not support direct switching from DMA to PT domain.
+ */
+ if (global_dirty_tracking && memory_region_is_ram(section->mr)) {
+ entry.iova = iova;
+ entry.translated_addr = iova;
In an experiment involving a nested setup with L1 + intel_iommu,
an L2 VM with an assigned igb VF can trigger the following QEMU
segv :
bitmap_set_atomic(map=NULL, start=1, nr=1)
physical_memory_set_dirty_range(start=0x380004040000, length=4096)
physical_memory_set_dirty_lebitmap(start=0x380004040000, pages=3)
vfio_container_query_dirty_bitmap(translated_addr=0x380004040000)
vfio_legacy_dma_unmap_one(iova=0x380004040000, size=12288)
vfio_listener_region_del()
Hmm, looks a range in PCI space, I suspect it's a PCI bar but it has 3 pages.
It's the nested VT-d interrupt remapping table.
Is igb VF a real hw passthrough to L1, then L1 pass it to L2?
Iit's an L1 emulated igb VF assigned to an L2.
Seems L2 crash, not L1?
yes. It occured a few times and I can't reproduce anymore. hmm.
It can be reproduced with below steps:
1. Start guest with VFIO device in legacy iommu mode, then start dirty rate
tracking.
(qemu) calc_dirty_rate -b 60
2. disable VFIO device bar region by clear bit1 in COMMAND register in guest.
# setpci -s 01:00.0 COMMAND=0005
3. below trace triggers:
vfio_listener_region_del region_del 0xe0400020000 - 0xe0400021fff
Thread 10 "CPU 6/KVM" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7ffde5dfb640 (LWP 217374)]
0x000055555613c8a0 in bitmap_set_atomic (map=0x0, start=32, nr=1) at
../util/bitmap.c:213
213 qatomic_or(p, mask_to_set);
(gdb) bt
#0 0x000055555613c8a0 in bitmap_set_atomic (map=0x0, start=32, nr=1) at
../util/bitmap.c:213
#1 0x0000555555d6a0e2 in physical_memory_set_dirty_range
(start=15410342789120, length=4096, mask=5 '\005') at ../system/physmem.c:1042
#2 0x0000555555d6b2c9 in physical_memory_set_dirty_lebitmap
(bitmap=0x7ffdc80ac690, start=15410342789120, pages=2) at
../system/physmem.c:1305
#3 0x0000555555ca97ee in vfio_legacy_dma_unmap_get_dirty_bitmap
(container=0x5555576fc980, iova=15410342789120, size=8192,
iotlb=0x7ffde5df9e20) at ../hw/vfio/container-legacy.c:112
#4 0x0000555555ca9927 in vfio_legacy_dma_unmap_one (container=0x5555576fc980,
iova=15410342789120, size=8192, flags=0, iotlb=0x7ffde5df9e20) at
../hw/vfio/container-legacy.c:145
#5 0x0000555555ca9aba in vfio_legacy_dma_unmap (bcontainer=0x5555576fc980,
iova=15410342789120, size=8192, iotlb=0x7ffde5df9e20, unmap_all=false) at
../hw/vfio/container-legacy.c:196
#6 0x0000555555ca84b5 in vfio_container_dma_unmap (bcontainer=0x5555576fc980,
iova=15410342789120, size=8192, iotlb=0x7ffde5df9e20, unmap_all=false) at
../hw/vfio/container.c:131
#7 0x0000555555ca68cd in vfio_listener_region_del (listener=0x5555576fc9b0,
section=0x7ffde5df9ed0) at ../hw/vfio/listener.c:753
#8 0x0000555555d5d9a9 in address_space_update_topology_pass (as=0x5555574c1880
<address_space_memory>, old_view=0x7ffd7c001580, new_view=0x7ffdc8000c00,
adding=false)
That seemed to fix it
- entry.translated_addr = iova;
+ entry.translated_addr = memory_region_get_ram_addr(section->mr) +
+ section->offset_within_region;
I verified this works.
Thanks,
I will send a patch.
C.
