Alex, thanks for you response. I tried to create guest with multiple root ports, but qemu report error: # qemu-system-x86_64 --accel kvm -m 4096 -smp 4 -drive file=/home/hao/rhel8.2.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -device virtio-net-pci,netdev=nic0,mac=00:16:3e:0d:e4:ab -netdev tap,id=nic0,br=virbr0,helper=/usr/local/libexec/qemu-bridge-helper -cpu host -machine q35 -device pcie-root-port,id=root1 -device pcie-root-port,id=root2 -daemonize qemu-system-x86_64: -device pcie-root-port,id=root2: Can't add chassis slot, error -16
Is the command wrong? -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/1921444 Title: Q35 doesn't support to hot add the 2nd PCIe device to KVM guest Status in QEMU: New Bug description: KVM: https://git.kernel.org/pub/scm/virt/kvm/kvm.git branch: next, commit: 4a98623d Qemu: https://git.qemu.org/git/qemu.git branch: master, commit: 9e2e9fe3 Created a KVM guest with Q35 chipset, and try to hot add 2 PCIe device to guest with qemu internal command device_add, the 1st device can be added successfully, but the 2nd device failed to hot add. If guest chipset is legacy i440fx, the 2 device can be added successfully. 1. Enable VT-d in BIOS 2. load KVM modules in Linux OS: modprobe kvm; modprobe kvm_intel 3. Bind 2 device to vfio-pci echo 0000:b1:00.0 > /sys/bus/pci/drivers/i40e/unbind echo "8086 1572" > /sys/bus/pci/drivers/vfio-pci/new_id echo 0000:b1:00.1 > /sys/bus/pci/drivers/i40e/unbind echo "8086 1572" > /sys/bus/pci/drivers/vfio-pci/new_id 4. create guest with Q35 chipset: qemu-system-x86_64 --accel kvm -m 4096 -smp 4 -drive file=/home/rhel8.2.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -cpu host -machine q35 -device pcie-root-port,id=root1 -daemonize 5. hot add the 1st device to guest successfully in guest qemu monitor "device_add vfio-pci,host=b1:00.0,id=nic0,bus=root1" 6. hot add the 2nd device to guest in guest qemu monitor "device_add vfio-pci,host=b1:00.1,id=nic1,bus=root1" The 2nd device doesn't be added in guest, and the 1st device is removed from guest. Guest partial log: [ 110.452272] pcieport 0000:00:04.0: pciehp: Slot(0): Attention button pressed [ 110.453314] pcieport 0000:00:04.0: pciehp: Slot(0) Powering on due to button press [ 110.454156] pcieport 0000:00:04.0: pciehp: Slot(0): Card present [ 110.454792] pcieport 0000:00:04.0: pciehp: Slot(0): Link Up [ 110.580927] pci 0000:01:00.0: [8086:1572] type 00 class 0x020000 [ 110.582560] pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x007fffff 64bit pref] [ 110.583453] pci 0000:01:00.0: reg 0x1c: [mem 0x00000000-0x00007fff 64bit pref] [ 110.584278] pci 0000:01:00.0: reg 0x30: [mem 0x00000000-0x0007ffff pref] [ 110.585051] pci 0000:01:00.0: Max Payload Size set to 128 (was 512, max 2048) [ 110.586621] pci 0000:01:00.0: PME# supported from D0 D3hot D3cold [ 110.588140] pci 0000:01:00.0: BAR 0: no space for [mem size 0x00800000 64bit pref] [ 110.588954] pci 0000:01:00.0: BAR 0: failed to assign [mem size 0x00800000 64bit pref] [ 110.589797] pci 0000:01:00.0: BAR 6: assigned [mem 0xfe800000-0xfe87ffff pref] [ 110.590703] pci 0000:01:00.0: BAR 3: assigned [mem 0xfe000000-0xfe007fff 64bit pref] [ 110.592085] pcieport 0000:00:04.0: PCI bridge to [bus 01] [ 110.592755] pcieport 0000:00:04.0: bridge window [io 0x1000-0x1fff] [ 110.594403] pcieport 0000:00:04.0: bridge window [mem 0xfe800000-0xfe9fffff] [ 110.595847] pcieport 0000:00:04.0: bridge window [mem 0xfe000000-0xfe1fffff 64bit pref] [ 110.597867] PCI: No. 2 try to assign unassigned res [ 110.597870] release child resource [mem 0xfe000000-0xfe007fff 64bit pref] [ 110.597871] pcieport 0000:00:04.0: resource 15 [mem 0xfe000000-0xfe1fffff 64bit pref] released [ 110.598881] pcieport 0000:00:04.0: PCI bridge to [bus 01] [ 110.600789] pcieport 0000:00:04.0: BAR 15: assigned [mem 0x180000000-0x180bfffff 64bit pref] [ 110.601731] pci 0000:01:00.0: BAR 0: assigned [mem 0x180000000-0x1807fffff 64bit pref] [ 110.602849] pci 0000:01:00.0: BAR 3: assigned [mem 0x180800000-0x180807fff 64bit pref] [ 110.604069] pcieport 0000:00:04.0: PCI bridge to [bus 01] [ 110.604941] pcieport 0000:00:04.0: bridge window [io 0x1000-0x1fff] [ 110.606237] pcieport 0000:00:04.0: bridge window [mem 0xfe800000-0xfe9fffff] [ 110.607401] pcieport 0000:00:04.0: bridge window [mem 0x180000000-0x180bfffff 64bit pref] [ 110.653661] i40e: Intel(R) Ethernet Connection XL710 Network Driver [ 110.654443] i40e: Copyright (c) 2013 - 2019 Intel Corporation. [ 110.655314] i40e 0000:01:00.0: enabling device (0140 -> 0142) [ 110.672396] i40e 0000:01:00.0: fw 6.0.48442 api 1.7 nvm 6.01 0x800035b1 1.1747.0 [8086:1572] [8086:0008] [ 110.750054] i40e 0000:01:00.0: MAC address: 3c:fd:fe:c0:59:98 [ 110.751792] i40e 0000:01:00.0: FW LLDP is enabled [ 110.764644] i40e 0000:01:00.0 eth1: NIC Link is Up, 10 Gbps Full Duplex, Flow Control: None [ 110.779390] i40e 0000:01:00.0: PCI-Express: Speed 8.0GT/s Width x8 [ 110.789841] i40e 0000:01:00.0: Features: PF-id[0] VFs: 64 VSIs: 66 QP: 4 RSS FD_ATR FD_SB NTUPLE DCB VxLAN Geneve PTP VEPA [ 111.817553] IPv6: ADDRCONF(NETDEV_CHANGE): eth1: link becomes ready [ 205.130288] pcieport 0000:00:04.0: pciehp: Slot(0): Attention button pressed [ 205.131743] pcieport 0000:00:04.0: pciehp: Slot(0): Powering off due to button press [ 205.133233] pcieport 0000:00:04.0: pciehp: Slot(0): Card not present [ 205.135728] i40e 0000:01:00.0: i40e_ptp_stop: removed PHC on eth1 To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1921444/+subscriptions
