On Fri, 2 Oct 2026, Paolo Bonzini wrote:
On 10/1/26 14:21, Peter Maydell wrote:
On Wed, 30 Sept 2026 at 09:54, Paolo Bonzini <[email protected]> wrote:
+- large, heavily LLM-assisted contributions where LLMs write large parts
+ of functional code. These are only allowed if *pre-arranged*,
+ *high-quality* and *well-tested*.
+
+The last case (*"LLM-created contribution"*) is the one that requires
+a more careful approach. In particular:
+
+- "Pre-arranged" means that a maintainer has communicated ahead of time,
+ for example on `IRC or the QEMU mailing
+ list <https://www.qemu.org/contribute/>`__, that they are willing
+ to review an LLM-created contribution.
+
+- "High-quality" means that the contributor must apply the same judgment
+ that would be applied to other code changes. Contributors must invest
+ substantial time reviewing their contributions, curating them, and
+ understanding them in depth.
+
+- "Well-tested" means the LLM-created contributions will be held to a
+ higher standard than human-created ones, because LLMs make it easier
+ to write tests. There are no exceptions for "writing the tests seems
+ hard" or for `yak shaving
<https://en.wiktionary.org/wiki/yak_shaving>`__.
...but I think fundamentally I am not in favour of having this
fourth category of permitted uses. It seems even to have lost the
idea of it being an "experimental" approach in this revision of the
series. The first three categories are fine, and I have no particular
problem with our relaxing our current policy to explicitly permit them.
Our problem as a project does not seem to be on the "writing
code" side of things, but has always been on the "code review"
side. This policy relaxation makes it much easier to write more
code. Even when there's a "pre-arranged reviewer" that isn't
necessarily helpful -- if you're one of the primary maintainers
for an area of QEMU then the fact that somebody else is reviewing
it doesn't make a big AI-generated code infusion any more palatable.
I understand where you're coming from.
The purpose of the fourth category is norming contributions that are already
happening. I apologize for the allegation in advance, but take for example
https://lore.kernel.org/qemu-devel/[email protected]/;
on one hand it is very high quality, on the other hand I give it at least a
70% likelihood of being heavily LLM-assisted for the implementation and not
just for the analysis.
The question is: do we want contributions like that one? If we decide that
their quality and usefulness are high enough to want them, what process do we
need around them? I think we all agree that, if that kind of contribution is
desirable, we want the submitters to declare AI usage and the reviewers to be
aware of it.
Allowing large heavily LLM-assisted contributions seems to me:
* the biggest risk legally speaking (though I think this is
in practice not all that big a risk at this point)
* the biggest risk of the codebase degenerating into
unmaintainability
* the biggest risk of finding ourselves reviewing contributions
that are in practice much more vibe-coded than the policy text
above is asking for
I agree completely with how you phrased the first and third. I agree that
the second bullet is a risk too, but it's mitigated by "LLM-created
contributions will be held to a higher standard than human-created ones,
because LLMs make it easier to write tests" (and by maintainers being able to
Tests need to be reviewed too to be able to trust them so that does not
help against overloading reviewers with generated patches but burdens them
even more: they will also have to review the tests before the code.
We could say tests must be written by hand but nobody wants to do that or
that they need to be reviewed by a human before submission but we ask that
for all code in the patch including tests so I don't see how the argument
that LLMs can write tests would reduce the original problem.
Regards,
BALATON Zoltan
