On Wed, 2026-09-02 at 13:01 +0100, Peter Maydell wrote: > On Tue, 1 Sept 2026 at 17:14, Paolo Bonzini <[email protected]> > wrote: > > > > Until now QEMU's code provenance policy declined any contribution > > believed to include or derive from AI-generated content. A blanket > > ban > > was easy to maintain while LLM output was rarely usable on its own, > > but > > as the tools improved an absolute prohibition has become harder to > > justify and enforce. > > Yes, but the "enforcement" part is just as hard to deal with for > any AI policy short of "we don't care whether AI was used or not > and don't care if it's labelled that way or not". A contributor > who was unaware of or chose to deliberately ignore our existing > AI policy is just as likely to be unaware of or to ignore an updated > policy that says "yes, but only with limitations and must be > pre-arranged for larger amounts of generated code". (A contributor > who deliberately ignores the policy is I hope going to be rare; > contributors who are simply unaware of it are something that we can > hopefully improve on by (a) having an AGENTS.md that will tell their > LLM about it and (b) asking in case of doubt on the maintainer side.)
Adding an AGENTS.md is a big part of this. I do think that it can be harder to enforce a strict policy as well. Like with most things, a blanket ban just leads people to ignore the ban and use AI anyway and lie about it. If we provide some leniency then people hopefully will be more likely to correctly declare usage and pre-arrange large patchsets. Which is a better situation then everyone just pretending their AI code isn't AI. > > I think fundamentally my issue here is that to me it doesn't seem > like the project has problems with "we aren't writing code fast > enough"; the problems are more "we don't have enough code review", > "our existing code is getting holes poked in it by automated > bug finding" and "we have too much old code that's barely used and > unmaintained". So an AI policy update whose major move is "allow > more generated code" feels bad to me, because it's inevitably > going to increase the workload on reviewers and maintainers. > I would be much happier seeing more use of AI in the project to > help us where we're struggling. > > > This proposed policy seeks to implement the cautious > > experimentation > > approach, inspired mostly by the Rust project's policy and by > > Software > > Freedom Conservancy's own recommendations on LLMs and generative > > AI. > > The question I had with the Rust policy was to what extent their > "experimentation" is actually going to be cautious experimentation, > and how much it turns out to be "this is a back door that lets the > people who want to write and review LLM-generated code do that anyway > as long as the people who don't want to can mostly ignore it". > > If this is really experimental then we should have some clear > boundaries and criteria for our experimentation, covering for > instance > (a) when do we assess the success or failure of the experiment? > (b) what are we looking at to decide success/failure? > (c) what are our boundaries for what kinds of change we're willing > to make under this experiment and what we are not? > (d) what is our plan for rolling back or otherwise recovering if > the experiment seems to have failed? > > The Rust policy makes some attempts at some of these e.g. with its > "circuit breaker" provision and the requirement that LLM changes are > "non-critical" ones. > > > +LLM-assisted and LLM-created contributions > > +'''''''''''''''''''''''''''''''''''''''''' > > + > > +Use of generative AI tools for code contributions generally falls > > into > > +four buckets: > > + > > +- "background" assistance, such as completion and code review. > > This does > > + not need disclosure, however you are expected to examine and > > understand > > + any code or any changes produced by the tool. > > + > > +- small LLM-assisted bugfixes (as a rule of thumb, below 10 lines > > of code) > > + are allowed with disclosure. Splitting a submission in multiple > > parts > > + is **not** a valid way to bypass this rule; repeated, frequent > > posting > > + of such fixes must follow the rule for full contributions. > > + > > +- use of LLMs to help generating parts of a larger patch---a test > > case, a > > + parser, boilerplate code for a new API, a tool to help > > performing > > + mechanical changes, etc. These are generally allowed, but > > disclosure > > + is recommended. > > I guess I'm generally OK with these (though I might add an "If in > doubt > about whether your use here is too extensive, ask" to the last one: > "a parser" is potentially a pretty big thing to be delegating to the > LLM, > for instance, and might either be "mostly boilerplate" or to shade > over > into the "writing large parts of functional code" category). > > I do note that even for "small bugfixes below 10 lines of code" the > code review effort can still be pretty huge where it's touching > something > like a device model, where you have to go and find the right > datasheet > or spec and confirm whether the proposed change is really the right > one > or if it just fixes whatever the assert/crash was but in the wrong > way. > (Handwritten example from me: I tried to fix an IDE divide-by-zero > https://patchew.org/QEMU/[email protected]/ > and even though I had read the IDE spec and the fix is pretty small > and > it does fix the bug's test case, it turns out that it's a lot more > complicated than that and the proper fix is something entirely > different.) > > Having a hundred "fix minor bug in old code" patches on the list that > are unreviewed isn't a lot better than having a hundred issues in the > bug tracker (indeed, it's arguably worse, since we have no tracking > system for patches on the mailing list; at least the issues won't > just > get lost in the deluge...) > > > +- large, heavily LLM-assisted contributions where LLMs write large > > parts > > + of functional code. These are only allowed if *pre-arranged*, > > + *high-quality* and *well-tested*. > > + > > +The last case (*"LLM-created contribution"*) is the one that > > requires > > +a more careful approach. In particular: > > + > > +- "Pre-arranged" means that a maintainer has communicated ahead of > > time, > > + for example on `IRC or the QEMU mailing > > + list <https://www.qemu.org/contribute/>`__, that they are > > willing > > + to review an LLM-created contribution. > > + > > +- "High-quality" means that the contributor must apply the same > > judgment > > + that would be applied to other code changes. Contributors must > > invest > > + substantial time reviewing their contributions, curating them, > > and > > + understanding them in depth; in particular, you are still > > expected to > > + :ref:`understand and explain your changes > > + <write_a_meaningful_commit_message>` and the rationale behind > > them. > > + > > +- "Well-tested" means the LLM-created contributions will be held > > to a > > + higher standard than human-created ones, because LLMs make it > > easier > > + to write tests. There are no exceptions for "writing the tests > > seems > > + hard" or for `yak shaving > > <https://en.wiktionary.org/wiki/yak_shaving>`__. > > This last part is the one that I am much less certain about, and > where > I'm not sure we have sufficiently bounded and defined our experiment > (as noted above). This is already happening today though. At least hopefully this prods people to declare it is > > I'm also more willing to allow leeway and to trust the judgement on > LLM use for somebody who is already a regular contributor to the > project (and so has some idea of how the codebase works and better > ability to spot when generated code has gone off in the wrong > direction), versus patches from somebody who hasn't contributed > before. Is that something we want to try to encode in policy > (e.g. with limitations on the "pre-arranged larger contribution" > case) ? That's the case for all submissions though, I think that's just how things work Alistair > > -- PMM
