Andreas Färber <afaer...@suse.de> writes: > Hi Markus, > > Again thanks for digging into this. > > Am 29.01.2015 um 15:58 schrieb Markus Armbruster: >> This test does everything a number of existing tests currently do: >> >> ac97-test.c e1000-test.c es1370-test.c eepro100-test.c >> ne2000-test.c nvme-test.c pcnet-test.c rtl8139-test.c >> tpci200-test.c virtio-balloon-test.c virtio-rng-test.c >> vmxnet3-test.c >> >> They are all marked "TODO: Replace with functional tests". Options >> >> * Delete them now, undelete when we add functional tests >> >> * Keep them, blacklist the devices in pci-devs-test.c >> >> * Live with the duplicated testing >> >> Andreas, I guess you got an opinion here. > > My preference would be to not remove device files. "Functional tests" > refers to doing real device-specific MMIO or PIO, and the intent of > contributing such stubs, beyond the basic -device init/realize testing, > was lowering the hurdle so that maintainers can require bugfixers to > contribute a matching test case where applicable.
Yes, asking people who aren't familiar with device tests to create one from scratch is asking a lot. Asking them to copy a skeleton and flesh it out is much more practical. Almost as good as fleshing out one of your stubs, I think. A non-skeleton test for a really simple PCI device could serve as skeleton. wdt_i6300esb.c is a possible candidate. >> There's overlap with a few others: >> >> i82801b11-test.c usb-hcd-ehci-test.c usb-hcd-ohci-test.c >> usb-hcd-xhci-test.c virtio-blk-test.c virtio-net-test.c >> virtio-scsi-test.c virtio-serial-test.c >> >> Options: >> >> * Blacklist the devices in pci-devs-test.c >> >> * Live with the duplicated testing >> >> Andreas? > > Not having reviewed the respective test code yet, I would suggest to > keep generic tests in pci-devs-test.c and to rather drop generic tests > from device-specific files (de-duplication). Makes sense to me. However, some existing device-specific files won't have any tests left then. What do you want me to do with them? > While I haven't benchmarked it, I assume that the bigger contributor to > qtest runtime is the overhead of spawning qemu-system-* processes rather > than running multiple tests per process. So living with duplication may > be a convenience option. My new test is a pig in that regard: it runs qemu-system-FOO once per non-blacklisted PCI device. I could pack tests into fewer runs by using more than just a single PCI slot. i440FX can do 30 without a pci-bridge (would probably a bad idea for this test) and multifunction (certainly a bad idea). However, when a test fails, the culprit isn't as obvious then. Do you want me to try that?