On Mon, Aug 12, 2024 at 9:22 AM Daniel P. Berrangé <berra...@redhat.com> wrote: > > On Mon, Aug 12, 2024 at 05:10:52PM +0100, Peter Maydell wrote: > > On Mon, 5 Aug 2024 at 21:17, Octavian Purdila <ta...@google.com> wrote: > > > It also introduces unit tests for device models. To allow accessing > > > registers from unit tests a system bus mock is created. The main > > > advantage of unit tests for device models over QTest is that device > > > models can be tested in isolation and do not require a full qemu > > > machine. > > > > On the other hand the disadvantage is that you need to add a > > bunch of extra code to mock the interfaces that the device > > connects to, and then you compile into the test binary a > > subset of C files which weren't written with the expectation > > that they got compiled into tests like that, so it feels a > > bit brittle. The nice thing about qtest is that it doesn't > > need you to do any of that -- you just run the QEMU machine > > model and prod the devices it already has. > > Yep, mocking often ends up being rather a double-edged sword. > You can do some really powerful things with it, and in particular > it can enable you test scenarios / code paths that are hard or > impractical to exercise with regular functional tests where fault > injection is difficult/impossible. > > The cost of this though is that the mocks can pose a significant > ongoing maint burden on anyone who refactors code in future, as > the mocks need to evolve in lockstep with the code refactoring. > > 12 months down the line, it can be hard for another maintainer > to understand the full purpose of the mocks & thus even harder > to know what todo as they refactor the code. The future burden > of mocks rarely falls on the person who creates them in the > first place. >
I agree that there are potentially maintenance issues. For what is worth, AFAIR, we only ran into one breaking change in about 3 years for which we used this approach which was also simple to fix (gpio was removed from hwcore so we had to pull in gpio.c directly). > > Do you have examples in this series of tests which you > > were able to write with this unit test setup that you > > wouldn't have been able to write equivalents of with the > > qtest framework ? > No, I think all of the tests in the patch set could be written in qtest.
