Hi all, In our test, we configured VM with several pci-bridges and a virtio-net nic been attached with bus 4, After VM is startup, We ping this nic from host to judge if it is working normally. Then, we hot add pci devices to this VM with bus 0. We found the virtio-net NIC in bus 4 is not working (can not connect) occasionally, as it kick virtio backend failure with error below: Unassigned mem write 00000000fc803004 = 0x1
memory-region: pci_bridge_pci 0000000000000000-ffffffffffffffff (prio 0, RW): pci_bridge_pci 00000000fc800000-00000000fc803fff (prio 1, RW): virtio-pci 00000000fc800000-00000000fc800fff (prio 0, RW): virtio-pci-common 00000000fc801000-00000000fc801fff (prio 0, RW): virtio-pci-isr 00000000fc802000-00000000fc802fff (prio 0, RW): virtio-pci-device 00000000fc803000-00000000fc803fff (prio 0, RW): virtio-pci-notify <- io mem unassigned … We caught an exceptional address changing while this problem happened, show as follow: Before pci_bridge_update_mappings: 00000000fc000000-00000000fc1fffff (prio 1, RW): alias pci_bridge_pref_mem @pci_bridge_pci 00000000fc000000-00000000fc1fffff 00000000fc200000-00000000fc3fffff (prio 1, RW): alias pci_bridge_pref_mem @pci_bridge_pci 00000000fc200000-00000000fc3fffff 00000000fc400000-00000000fc5fffff (prio 1, RW): alias pci_bridge_pref_mem @pci_bridge_pci 00000000fc400000-00000000fc5fffff 00000000fc600000-00000000fc7fffff (prio 1, RW): alias pci_bridge_pref_mem @pci_bridge_pci 00000000fc600000-00000000fc7fffff 00000000fc800000-00000000fc9fffff (prio 1, RW): alias pci_bridge_pref_mem @pci_bridge_pci 00000000fc800000-00000000fc9fffff <- correct Adress Spce 00000000fca00000-00000000fcbfffff (prio 1, RW): alias pci_bridge_pref_mem @pci_bridge_pci 00000000fca00000-00000000fcbfffff 00000000fcc00000-00000000fcdfffff (prio 1, RW): alias pci_bridge_pref_mem @pci_bridge_pci 00000000fcc00000-00000000fcdfffff 00000000fce00000-00000000fcffffff (prio 1, RW): alias pci_bridge_pref_mem @pci_bridge_pci 00000000fce00000-00000000fcffffff After pci_bridge_update_mappings: 00000000fda00000-00000000fdbfffff (prio 1, RW): alias pci_bridge_mem @pci_bridge_pci 00000000fda00000-00000000fdbfffff 00000000fdc00000-00000000fddfffff (prio 1, RW): alias pci_bridge_mem @pci_bridge_pci 00000000fdc00000-00000000fddfffff 00000000fde00000-00000000fdffffff (prio 1, RW): alias pci_bridge_mem @pci_bridge_pci 00000000fde00000-00000000fdffffff 00000000fe000000-00000000fe1fffff (prio 1, RW): alias pci_bridge_mem @pci_bridge_pci 00000000fe000000-00000000fe1fffff 00000000fe200000-00000000fe3fffff (prio 1, RW): alias pci_bridge_mem @pci_bridge_pci 00000000fe200000-00000000fe3fffff 00000000fe400000-00000000fe5fffff (prio 1, RW): alias pci_bridge_mem @pci_bridge_pci 00000000fe400000-00000000fe5fffff 00000000fe600000-00000000fe7fffff (prio 1, RW): alias pci_bridge_mem @pci_bridge_pci 00000000fe600000-00000000fe7fffff 00000000fe800000-00000000fe9fffff (prio 1, RW): alias pci_bridge_mem @pci_bridge_pci 00000000fe800000-00000000fe9fffff fffffffffc800000-fffffffffc800000 (prio 1, RW): alias pci_bridge_pref_mem @pci_bridge_pci fffffffffc800000-fffffffffc800000 <- Exceptional Adress Space We have figured out why this address becomes this value, according to pci spec, pci driver can get BAR address size by writing 0xffffffff to the pci register firstly, and then read back the value from this register. We didn't handle this value specially while process pci write in qemu, the function call stack is: Pci_bridge_dev_write_config -> pci_bridge_write_config -> pci_default_write_config (we update the config[address] value here to fffffffffc800000, which should be 0xfc800000 ) -> pci_bridge_update_mappings ->pci_bridge_region_del(br, br->windows); -> pci_bridge_region_init ->pci_bridge_init_alias (here pci_bridge_get_base, we use the wrong value fffffffffc800000) -> memory_region_transaction_commit So, as we can see, we use the wrong base address in qemu to update the memory regions, though, we update the base address to The correct value after pci driver in VM write the original value back, the virtio NIC in bus 4 may still sends net packets concurrently with The wrong memory region address. We have tried to skip the memory region update action in qemu while detect pci write with 0xffffffff value, and it does work, but This seems to be not gently. diff --git a/hw/pci/pci_bridge.c b/hw/pci/pci_bridge.c index b2e50c3..84b405d 100644 --- a/hw/pci/pci_bridge.c +++ b/hw/pci/pci_bridge.c @@ -256,7 +256,8 @@ void pci_bridge_write_config(PCIDevice *d, pci_default_write_config(d, address, val, len); - if (ranges_overlap(address, len, PCI_COMMAND, 2) || + if ( (val != 0xffffffff) && + (ranges_overlap(address, len, PCI_COMMAND, 2) || /* io base/limit */ ranges_overlap(address, len, PCI_IO_BASE, 2) || @@ -266,7 +267,7 @@ void pci_bridge_write_config(PCIDevice *d, ranges_overlap(address, len, PCI_MEMORY_BASE, 20) || /* vga enable */ - ranges_overlap(address, len, PCI_BRIDGE_CONTROL, 2)) { + ranges_overlap(address, len, PCI_BRIDGE_CONTROL, 2))) { pci_bridge_update_mappings(s); } Thinks, Xu