On 12/28/2011 06:44 PM, Anthony Liguori wrote: > On 12/28/2011 09:28 AM, Avi Kivity wrote: >> On 12/28/2011 05:01 PM, Avi Kivity wrote: >>> I'd say that running a ping test is a weak version of kvm-autotest's >>> system tests. Running a synthetic test that pokes values into memory >>> and mmio and sees a packet coming out is a unit test (the latter can in >>> fact be executed without a guest at all, just a table driving calls to >>> the memory and irq APIs). >>> >> >> Consider >> 98d23704138e0 >> 7b4252e83f6f7d >> f7e80adf3cc4 >> c16ada980f43 >> 4abf12f4ea8 >> >> (found by looking for 'fix' in the commit log and filtering out the >> commits that don't support my case) >> >> how can you reject such patches on the grounds that they're not >> accompanied by unit tests? > > That's why I've also proposed qtest. But having written quite a few > qtest unit tests by now, you hit the limits of this type of testing > pretty quickly.
Can you describe those limits? > >> only by making it easy to add tests for >> them. I think it would be hard/impossible to test them with >> linux-as-a-guest, since they fix edge cases that linux doesn't invoke. >> But by having our own driver (often just using qmp to poke at memory), >> we can easily generate the sequence that triggers the error. >> >> We'd probably need a library to support setting up a pci device's BARs, >> but that's easy with qmp/python integration. You can even poke a small >> executable into memory and execute it directly, if you really need guest >> cpu interaction. > > Please review the qtest series. I think it offers a pretty good > approach to writing this style of test. But as I mentioned, you hit > the limits pretty quickly. I think it's great, it looks like exactly what I wanted, except it's been delivered on time. I'd really like to see it integrated quickly with some flesh around it, then replying -ENOTEST to all patches. This will improve qemu's quality a lot more than guest boot/ping tests, which we do regularly with kvm-autotest anyway. Think of how new instruction emulations are always accompanied by new kvm-unit-tests tests, I often don't even have to ask for them. -- error compiling committee.c: too many arguments to function