Hi Alex, 2014-07-30 22:46 GMT+08:00 Alex Williamson <alex.william...@redhat.com>: > On Wed, 2014-07-30 at 22:16 +0800, Le Tan wrote: >> Hi Michael, >> >> 2014-07-30 21:16 GMT+08:00 Michael S. Tsirkin <m...@redhat.com>: >> > On Wed, Jul 30, 2014 at 08:24:04PM +0800, Le Tan wrote: >> >> Hi, >> >> I am testing vfio in L1 with my VT-d emulation project. I assigned one >> >> of the two AHCI controllers in L1 to L2 via vfio. After I ran the QEMU >> >> in L1, it complains that: >> >> qemu-system-x86_64: vfio: Cannot reset device 0000:00:03.0, no >> >> available reset mechanism. >> >> qemu-system-x86_64: vfio: Cannot reset device 0000:00:03.0, no >> >> available reset mechanism. >> >> >> >> Then L2 paused when the SeaBIOS executed in ahci_controller_setup(). I >> >> look into this and found that: >> >> val = ahci_ctrl_readl(ctrl, HOST_CTL); >> >> ahci_ctrl_writel(ctrl, HOST_CTL, val | HOST_CTL_AHCI_EN); >> >> When the BIOS tried to read the HOST_CTL, it returns 0x80000002, which >> >> bit 2 (Interrupt Enable) is 1. The AHCI manual says that this bit >> >> should be cleared by default. So maybe L1 didn't reset the device >> >> before assigning it to L2? >> >> Then the BIOS tried to write back to HOST_CTL and it was stuck here. :( >> >> >> >> So can anyone give me some advice? About the state of PCI device or >> >> bus-level reset? >> >> >> >> Here is the detail of the environment and the way I did the vfio. >> >> 1. lspci in L1 said: >> >> 00:03.0 SATA controller [0106]: Intel Corporation 82801IR/IO/IH >> >> (ICH9R/DO/DH) 6 port SATA Controller [AHCI mode] [8086:2922] (rev 02) >> >> 00:1f.0 ISA bridge [0601]: Intel Corporation 82801IB (ICH9) LPC >> >> Interface Controller [8086:2918] (rev 02) >> >> 00:1f.2 SATA controller [0106]: Intel Corporation 82801IR/IO/IH >> >> (ICH9R/DO/DH) 6 port SATA Controller [AHCI mode] [8086:2922] (rev 02) >> >> 00:1f.3 SMBus [0c05]: Intel Corporation 82801I (ICH9 Family) SMBus >> >> Controller [8086:2930] (rev 02) >> >> 2. Unbind 00:03.0 and do vfio: >> >> modprobe -r vfio_iommu_type1 >> >> modprobe vfio_iommu_type1 allow_unsafe_interrupts=1 >> >> modprobe vfio-pci >> >> echo 0000:00:03.0 > /sys/bus/pci/devices/0000\:00\:03.0/driver/unbind >> >> echo "8086 2922" > /sys/bus/pci/drivers/vfio-pci/new_id >> >> 3. run L2 with "-device vfio-pci,host=00:03.0" >> >> >> >> Any help is appreciated! Thanks very much! >> >> >> >> Regards, >> >> Le >> > >> > Clearly, bus level reset can't work for the root bus :) >> >> Thanks very much! >> I test the vfio with a second-bus ahci controller and it didn't >> complain about the lack of reset mechanism. :) And the return val of >> HOST_CTL is normal now (the same as emulated ahci controller). >> However, it still paused when the BIOS tried to write to the HOST_CTL. >> Do you have any idea? >> And we should just test vfio and legacy pci-assignment with second bus >> devices, not considering the root-bus devices? > > AHCI seems like a poor choice of device for this work, they typically > don't support any kind of reset and they can be troublesome even for the > L1 assignment. You really want something with FLR support so that both > the host and L1 guest can potentially reset the device. That said, you > may still run into bugs with a L1 guest directed FLR. Thanks, > > Alex >
So what device do you think is suitable for the pci-assign test? e1000? I just tested it with sound card ac97 and USB controller. But I don't know how to attach them to a pcie-to-pci bridge, so maybe they weren't reset before being assigned to L2. But it seems that they can work. 1. With the sound card, I assigned it to L2 via both vfio and legacy pci-assign and I could hear the music played in L2 from host's speakers. Of course, the vfio also complained about the lack of reset mechanism. 2. With the USB controller, I used "-usb -usbdevice disk:file" to attach a USB disk to L1. But there were 4 related devices in L1, so I didn't know what should be assigned to L2. the info qtree was like this: dev: ich9-usb-uhci3, id "" masterbus = "usb-bus.0" firstport = 4 (0x4) bandwidth = 1280 (0x500) maxframes = 128 (0x80) addr = 1d.2 dev: ich9-usb-uhci2, id "" masterbus = "usb-bus.0" firstport = 2 (0x2) bandwidth = 1280 (0x500) maxframes = 128 (0x80) addr = 1d.1 dev: ich9-usb-uhci1, id "" masterbus = "usb-bus.0" firstport = 0 (0x0) bandwidth = 1280 (0x500) maxframes = 128 (0x80) addr = 1d.0 dev: ich9-usb-ehci1, id "" maxframes = 128 (0x80) addr = 1d.7 bus: usb-bus.0 type usb-bus dev: usb-storage, id "" drive = "" logical_block_size = 512 (0x200) physical_block_size = 512 (0x200) min_io_size = 0 (0x0) opt_io_size = 0 (0x0) bootindex = -1 (0xffffffffffffffff) discard_granularity = 4294967295 (0xffffffff) removable = false port = "" serial = "" full-path = true msos-desc = true addr 0.1, port 1, speed 480, name QEMU USB MSD, attached bus: scsi.0 type SCSI dev: scsi-disk, id "" drive = "usb0" logical_block_size = 512 (0x200) physical_block_size = 512 (0x200) min_io_size = 0 (0x0) opt_io_size = 0 (0x0) Then I tried to unbind and bind both 1d.7 and 1d.0. Then I assigned 1d.0 to L2. L2 could boot smoothly and I could mount and access the USB disk. So what do you think? Thanks very much. Le