Am 06.10.2026 um 11:56 hat Daniel P. Berrangé geschrieben: > On Tue, Oct 06, 2026 at 10:44:37AM +0100, Alex Bennée wrote: > > Kevin Wolf <[email protected]> writes: > > > > > Am 05.10.2026 um 23:09 hat BALATON Zoltan geschrieben: > > >> On Fri, 2 Oct 2026, Paolo Bonzini wrote: > > >> > I agree completely with how you phrased the first and third. I agree > > >> > that the second bullet is a risk too, but it's mitigated by > > >> > "LLM-created > > >> > contributions will be held to a higher standard than human-created > > >> > ones, > > >> > because LLMs make it easier to write tests" (and by maintainers being > > >> > able to > > >> > > >> Tests need to be reviewed too to be able to trust them so that does not > > >> help > > >> against overloading reviewers with generated patches but burdens them > > >> even > > >> more: they will also have to review the tests before the code. > > > > > > The required level of trust for tests is comparatively low, so it's much > > > easier to review tests to a sufficient level than say device emulation > > > code. > > > > > > If a new test looks plausible, fails before the fix and passes > > > afterwards, it's more useful that what we had before, even if it's > > > technically buggy in some corner case. > > > > And also easier to rip out in the unlikely event it infringes anything. > > > > But broadly having a test* is better than having no test. > > > > * modulo the effects on our coverage. > > The maintability of test code is really important too, just as much > as the code being tested. Every test potentially constrains how we > can change code in future as tests themselves are never perfect or > exhaustive and often take short-cuts in their design to suit the > current testing needs. As we enhance the code under test, the test > suite needs to evolve with it and that requires that we understand > the test code and why it is doing what it is doing. > > A test spat out by AI which is a string of 20 undocumented qtest > commands may fail before the fix and pass afterwards, but that > test is not automatically a win if we don't understand why the test > is doing what it is doing - it becomes a burden for long term maint.
That's the "looks plausible" part. It's really not a lot of work to recognise that a test is just 20 undocumented qtest commands. Kevin
