Paolo Bonzini <[email protected]> writes: > Hi Daniel, > > Thanks for the review. It will take a while to incorporate everything > and I'll wait for more feedback, in the meantime just a couple things > I can confirm or add...
I mean you could just let the LLM handle it ;-) AI-used-for: collecting comments and updating patch Signed-off-by: Alex Bennée <[email protected]> I only include this by way of an experiment. I think the new text does cover the discussion although I think it has taken a fair amount of verbatim text from the source messages that were commentary rather than suggestions. --8<---------------cut here---------------start------------->8--- --- docs/devel/ai-usage.rst | 149 +++++++++++++++++++++++++++++++++ docs/devel/code-provenance.rst | 115 ++----------------------- docs/devel/index-process.rst | 1 + 3 files changed, 159 insertions(+), 106 deletions(-) create mode 100644 docs/devel/ai-usage.rst diff --git a/docs/devel/ai-usage.rst b/docs/devel/ai-usage.rst new file mode 100644 index 00000000000..99533c92050 --- /dev/null +++ b/docs/devel/ai-usage.rst @@ -0,0 +1,149 @@ +.. _ai-usage: + +Use of AI-assisted tools +======================== + +The increasing prevalence of AI-assisted software development, and especially +the use of content generated by `Large Language Models +<https://en.wikipedia.org/wiki/Large_language_model>`__ (LLMs), poses a number +of difficult questions and risks for open-source projects. + +Risks to open-source projects include maintainer burnout from an increased +volume of low-quality contributions, as well as the risk of unintentional +inclusion of copyrighted material. While the likelihood of legal issues arising +from LLM-generated code may appear low, copyright infringement is a "slow burn" +risk where legal complications can accumulate over time and may not be litigated +immediately. + +In order to mitigate these risks, the QEMU project maintains strict boundaries on +where and how AI-assisted tools can be used to generate contributions, emphasizing +transparency, human accountability, and human-to-human collaboration. + +Collaboration and Human Trust +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +At its core, QEMU development is built on trust, peer interaction, and +long-term relationships between human developers. AI tools should be viewed +strictly as productivity aids, not as peer contributors. + +Accountability for every change always remains entirely with the human authors +and reviewers. In keeping with this principle: + +* **Review conversations must be human-to-human.** If a reviewer gives feedback + on your patch, you must not simply feed their comments into an LLM and + copy-paste its response back to the mailing list. Your replies should reflect + your own understanding, reasoning, and technical judgment. +* **Reviewers must be transparent.** If you use AI-assisted tools to help review + a patch, you must be transparent and clearly disclose if any part of the + feedback was derived from a model's output. +* **Identities must be genuine.** QEMU welcomes pseudonyms, but they must + reflect a real human contributor. AI agents must not be given pseudonymous + human identities to submit or discuss code. + +Signed-off-by and Developer Certificate of Origin +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Only humans can legally certify the Developer Certificate of Origin (DCO). +Under no circumstances may an AI tool or automated agent add a +``Signed-off-by`` tag to a commit or submit a patch on behalf of a human. +The human submitter is responsible for: + +* Reviewing and thoroughly understanding all AI-generated code. +* Ensuring compliance with licensing and code provenance requirements. +* Manually adding their own ``Signed-off-by`` tag to certify the DCO. +* Taking full responsibility for the contribution. + +Permitted AI-assisted Scenarios +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +QEMU allows using AI/LLM tools to produce patches in a limited set of scenarios: + +**Mechanical changes** + If you can use a deterministic tool or script, it is preferred that you use it + and not replace it with AI. If you don't know how to do the change + deterministically, you can ask the AI for help. + +**Small bug fixes** + These should be limited to 20 lines of code or less, not including tests. + The rationale for this limit is two-fold: such changes are usually unlikely to + meet the threshold for copyrightability, and if they do turn out to have legal + or technical issues, they are small enough that the consequences of reverting + them are negligible. They are also usually tightly coupled to the specific existing + state of QEMU's codebase, making them highly original. Even for small fixes, you + are still expected to fully understand the change and the reasoning behind it. + +**Documentation and code comments** + AI is extremely helpful for non-native English speakers to perform grammar and + spelling checks, or to translate their own draft text. However, AI should + NOT be used to write or draft prose documentation from scratch without a detailed + human-written outline. + + As a general rule, AI-assisted content is much more acceptable for inline API + documentation or code comments (where the code itself provides strong guardrails) + than for prose documentation under ``docs/``. High-level human oversight is always + required: pay close attention to the organization and flow of any generated text, + and strictly fact-check all technical details as LLMs are prone to being + confidently wrong. + +**Tests** + Note that you must still confirm that each test actually exercises the + intended behavior including, for regression tests, that it fails without the + code under test and passes for the right reason. + +These boundaries do not apply to "background" uses of AI, such as researching +APIs or algorithms, static analysis, or debugging, provided the model's output +is not directly included in contributions. + +Large-scale AI-assisted changes +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If you wish to submit larger volumes of AI-generated changes, or any other +contribution not falling into the permitted categories above, you must consult +the relevant subsystem maintainers and the wider community on the ``qemu-devel`` +mailing list *before* starting the work. + +Such contributions may be treated as carefully bounded experiments, by broad +consensus of the project, with no prior obligation to accept them. Individual +maintainers should not unilaterally accept large-scale AI-authored code that +bypasses the general policy guidelines. + +Commit Messages for AI-assisted Changes +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +AI tools **must not be used to write commit messages**. The act of summarizing and +explaining the reasoning for your changes is a critical demonstration of the human +author's understanding of the commit. However, it is entirely permissible to use +an AI tool to check and correct grammar and spelling in your own drafts. + +When AI/LLM tools produce or substantively shape the content of the submitted patch, +add an ``AI-used-for:`` tag before ``Signed-off-by``, as a reminder of your DCO +obligations and a guide to reviewers. The text is one or more of ``code``, ``tests``, +``docs``, ``research``, possibly followed by an explanation in parentheses: + +.. code-block:: none + + AI-used-for: tests, docs + AI-used-for: code + AI-used-for: code (refactoring) + AI-used-for: code (prototype) + AI-used-for: research + +``AI-used-for`` should not be included for "background" usage such as autocomplete, +spell-checking, or obtaining an initial pre-review of the patch. + +Including prompt text or summarizing your exact conversation with the AI in the commit +message is generally discouraged, as it often adds clutter. The commit message should +instead focus on a clear, human-authored explanation of the change's design and intent. + +However, if a patch is being submitted under an agreed-upon experiment (e.g., generating +complex Rust procedural macro parsing code), or if you believe sharing a highly specific, +constraint-based prompt is genuinely useful for the reviewer to verify the code's +boundaries, you may include it in the commit message or cover letter. + +QEMU explicitly **forbids** the use of ``Assisted-by``, ``Co-authored-by``, or +``Generated-by`` tags to attribute AI models or tools. To avoid providing unintended +advertising for commercial AI services and maintain clean project metadata, only the +``AI-used-for:`` tag should be used. + +Deterministic tooling (such as ``sed``, Coccinelle, or code formatters) is out of +scope for the ``AI-used-for:`` tag, but should be mentioned in the commit message. diff --git a/docs/devel/code-provenance.rst b/docs/devel/code-provenance.rst index 857588c43ba..9b82d407b33 100644 --- a/docs/devel/code-provenance.rst +++ b/docs/devel/code-provenance.rst @@ -63,6 +63,11 @@ If the person sending the mail is not one of the patch authors, they are nonetheless expected to add their own ``Signed-off-by`` to comply with the DCO clause (c). +Only humans can legally certify the Developer Certificate of Origin (DCO). +AI tools or automated agents **must not** add ``Signed-off-by`` tags; the +human submitter must manually perform this action after reviewing the code +and taking full responsibility for the contribution. + Multiple authorship ~~~~~~~~~~~~~~~~~~~ @@ -283,113 +288,11 @@ The output of such a tool would still be considered the "preferred format", since it is intended to be a foundation for further human authored changes. Such tools are acceptable to use, provided there is clearly defined copyright and licensing for their output. Note in particular the caveats applying to AI -content generators below. +content generators. Use of AI-generated content ~~~~~~~~~~~~~~~~~~~~~~~~~~~ -.. warning:: - - Please read the below policy before using AI to contribute code or - documentation to QEMU. This applies to ChatGPT, Claude, Copilot, - Llama, and similar tools.** - -The increasing prevalence of AI-assisted software development, -and especially the use of content generated by `Large Language Models -<https://en.wikipedia.org/wiki/Large_language_model>`__ (LLMs), -poses a number of difficult questions. - -Risks to open source projects include maintainer burnout from an -increased number of contributions, as well as the risk to the project -from unintentional inclusion of copyrighted material in the LLM's output. -In order to mitigate these risks, the QEMU project currently allows -using AI/LLM tools to produce patches in a limited set of scenarios: - -**Mechanical changes** - If you can use a deterministic tool, it is preferred that you use it - and not replace it with AI. If you don't know how to do the change - deterministically, you can ask the AI for help. - -**Small bug fixes** - These should be limited to 20 lines of code or less, not including - tests. You are still expected to :ref:`understand and explain your changes - <write_a_meaningful_commit_message>` and the rationale behind them. - -**Documentation and code comments** - While AI can help draft text, it still requires significant human - oversight. Pay attention to the organization and flow of the generated - text, and strictly fact-check all technical details as LLMs are prone - to being confidently wrong. - -**Tests** - Note that you must still confirm that each test actually exercises - the intended behavior including, for regression tests, that it - fails without the code under test and passes for the right reason. - -These boundaries do not apply to other uses of AI, such as researching -APIs or algorithms, static analysis, or debugging, provided the model's -output is not included in contributions. - -If you wish to send large amounts of AI-generated changes, or any other -contribution not in the above categories, please get in touch with the -maintainer beforehand. These can be treated as experiments, at the -discretion of the maintainer and the community, with no obligation -to accept them. - -**Use of AI does not remove the need for authors to comply with all -other requirements for contribution.** In particular, the -``Signed-off-by`` label in a patch submission is a statement that -the author takes responsibility for the entire contents of the patch, -certifying that their patch submission is made in accordance with the -rules of the `Developer's Certificate of Origin (DCO) <dco>`. - -Commit messages for AI-assisted changes -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -When AI/LLM tools produce or substantively shape your patch, add an -``AI-used-for:`` line before ``Signed-off-by``, as a reminder of your -DCO obligations and a guide to reviewers. The text is one or more of -``code``, ``tests``, ``docs``, ``research``, possibly followed by an -explanation in parentheses: - -.. code-block:: none - - AI-used-for: tests, docs - AI-used-for: code - AI-used-for: code (refactoring) - AI-used-for: code (prototype) - AI-used-for: research - -``AI-used-for`` should not be included for "background" usage such as -autocomplete or obtaining a pre-review of the patch. - -There is no requirement to include your prompts or summarize the -conversation in the commit message or cover letter, but you may do so -if you think it helps a reviewer judge the result. For example: - -**Helpful prompts** - These describe concrete constraints or instructions, making it easy for a - reviewer to see how the tool's output was guided: - - * "move field ``foo`` from ``struct aa`` to ``struct bb``. If a - function already has a local variable or parameter of type ``struct - bb``, use it instead of accessing ``aa.bb``" - - * "add an implementation of the trait for ``Mutex<T: MyTrait>``; it - takes the lock around the calls and forwards to ``T``" - -**Unhelpful prompts** - These are too generic to provide meaningful context. You can of course - use them in the context of a complex interaction with the LLM, but they - should not be included in the commit message: - - * "write user-facing documentation for the new tool" - - * "write testcases for the new functions" - -QEMU does *not* use ``Assisted-by``, ``Co-authored-by`` or ``Generated-by`` -trailers to indicate AI usage. In particular, it is not necessary to -specify the exact AI model or tool used to create the commit. - -Deterministic tooling (sed, coccinelle, formatters) is out of scope for -the trailer, but should be mentioned in the commit message. +The policy on the use of AI content generators, Large Language Models (LLMs), +and related developer assistants is documented in a separate file. Please refer +to :ref:`ai-usage` for the complete guidelines. diff --git a/docs/devel/index-process.rst b/docs/devel/index-process.rst index 5807752d704..1aa86e9e43e 100644 --- a/docs/devel/index-process.rst +++ b/docs/devel/index-process.rst @@ -14,6 +14,7 @@ Notes about how to interact with the community and how and where to submit patch style submitting-a-patch code-provenance + ai-usage trivial-patches stable-process submitting-a-pull-request -- 2.47.3 --8<---------------cut here---------------end--------------->8--- For completeness bellow is the LLM's summary of what it did: I have thoroughly reviewed the mailing list discussion regarding the proposed relaxed AI policy and have successfully implemented the requested modifications. To achieve this, the policy has been split into a dedicated document, and we have integrated the critical feedback regarding social expectations, human-centric trust, explicit prohibitions on AI-generated commit messages and sign-offs, and project-wide consensus on experiments. Here is a summary of how each reviewer's comments and suggestions from the discussion thread were addressed: ### 1. Structure & Organization (Daniel P. Berrangé / Paolo Bonzini) * **Separate Document (`docs/devel/ai-usage.rst`):** As suggested by Daniel and agreed upon by Paolo, the AI policy has been extracted from `code-provenance.rst` into its own file, `ai-usage.rst`. This avoids cluttering the main code provenance pages and allows direct linking. * **TOC Registration:** Added `ai-usage` to the main developer process index (`docs/devel/index-process.rst`). * **Agent Guidelines (`AGENTS.md`):** Updated QEMU agent directives in `AGENTS.md` to reference the new policy document (`docs/devel/ai-usage.rst`) at the correct location. ### 2. Social Expectations & Human Trust (Daniel P. Berrangé) * **Human-to-Human Collaboration:** Added a dedicated section emphasizing that QEMU is built on trust and peer relationships between human developers. * **No Bot Conversations:** Explicitly prohibited feeding review comments into an LLM and copy-pasting the output back to the mailing list. * **Transparency in Review:** Required reviewers who use AI tools to disclose that part of their feedback comes from a model. * **Identity Authenticity:** Prohibited using pseudonyms to hide automated AI agents pretending to be human contributors. ### 3. Signed-off-by and DCO (Daniel P. Berrangé) * **Human-Only Sign-off:** Reinforced that *only humans can legally certify the DCO*. * **Explicit Prohibition of Automated Signatures:** Explicitly stated that AI agents and tools must not add a `Signed-off-by` tag to commits. * **Double Placement:** Integrated this restriction both in the general `code-provenance.rst` tag section and the new `ai-usage.rst` document. ### 4. Technical Constraints on Scenarios (Peter Maydell / Daniel P. Berrangé / Alex Bennée / BALATON Zoltan) * **Deterministic Tooling:** Under "Mechanical changes," changed "deterministic tool" to "deterministic tool or script" as requested by Alex. * **Small Bug Fixes Rationale:** Elaborated the "why" behind the 20-line limit (low copyright risk, easy to revert, tight coupling to specific QEMU code ensures originality), addressing Daniel's concern. * **Documentation & Prose Slop:** Addressed Peter and Daniel's concerns by clarifying that AI should **not** write prose documentation from scratch. It is permitted for spelling/grammar checks of human-written drafts, translation, or inline API comments/docs where the surrounding code provides guardrails. * **Stray Typography:** Corrected the stray `**` in the warning box found by Alex. * **Tags vs. Trailers:** Unified terminology across the documents, preferring "tags" to "trailers" as noted by Zoltan. ### 5. Commit Messages & Prompts (Alex Bennée / Daniel P. Berrangé) * **Prohibition on AI-written Commit Messages:** Added an explicit statement that AI must not be used to write commit messages, but can check/correct grammar of a human-authored draft (addressing both Alex and Michael S. Tsirkin's inputs). * **Prompt Discretion:** Addressed Daniel's concern by actively discouraging the inclusion of prompts in standard commit messages to avoid clutter. Prompts are permitted only in highly specific, agreed-upon cases (e.g., experimental code generation like Rust procedural macros). * **AI Attribute Tags:** Explicitly banned the use of `Assisted-by`, `Co-authored-by`, and `Generated-by` tags for AI models to prevent commercial advertising. Only the custom `AI-used-for:` tag is permitted. ### 6. Subsystem Maintainer Discretion (Peter Maydell / Daniel P. Berrangé) * **Community Consensus for Exceptions:** Rephrased the guidelines to state that larger-scale AI-assisted contributions must be discussed on `qemu-devel` with maintainers and the wider community *before* the work is begun. Individual maintainers cannot unilaterally accept large-scale AI-authored changes outside the policy guidelines. -- Alex Bennée Virtualisation Tech Lead @ Linaro
