While triaging the issue tracker I wondered if this would be a suitable job for an AI agent. Unfortunately the OSS program doesn't give any credits to run agents in gitlab. However I do have access to models from my editor and ECA so I built one and tested it on a few issues.
We might want to consider adding some common "issue smells" for common problems in issues. For example triggering asserts that have been added to the code and other such annoyances. Signed-off-by: Alex Bennée <[email protected]> --- v4 - fix a bunch of typos in SKILL - update evals - remove mentions of ECA - fix typo referencing old skill name - remove eca__spawn_agent reference - clean-up authentication handling - teach it about scoped labels - make less chatty in the comments - updated commit msg - suggest running things in parallel (use agents) - more pointers to incompleteness v3 - more imperative phrasing v2 - prefer sub-agent - mention parameters - don't be over eager to tag host or target for generic bugs - signoff comments as an agent on behalf of the user - detect patch available and apply appropriate tags --- .agents/skills/qemu-issue-triage/SKILL.md | 112 +++++++++++++++ .../qemu-issue-triage/assets/labels.txt | 133 ++++++++++++++++++ .../skills/qemu-issue-triage/evals/evals.json | 18 +++ .../scripts/update_labels.sh | 16 +++ AGENTS.md | 1 + 5 files changed, 280 insertions(+) create mode 100644 .agents/skills/qemu-issue-triage/SKILL.md create mode 100644 .agents/skills/qemu-issue-triage/assets/labels.txt create mode 100644 .agents/skills/qemu-issue-triage/evals/evals.json create mode 100755 .agents/skills/qemu-issue-triage/scripts/update_labels.sh diff --git a/.agents/skills/qemu-issue-triage/SKILL.md b/.agents/skills/qemu-issue-triage/SKILL.md new file mode 100644 index 00000000000..a0802e24dd3 --- /dev/null +++ b/.agents/skills/qemu-issue-triage/SKILL.md @@ -0,0 +1,112 @@ +--- +name: qemu-issue-triage +description: Use this skill to triage and label GitLab issues for the QEMU project +license: GPL-2.0-or-later +--- + +# Instructions + +This skill provides specialized instructions for triaging GitLab issues for the QEMU project. + +## Parameter Handling & Execution Strategy (CRITICAL) +1. **Parameters**: If the user invokes this skill with multiple arguments or a list of IDs (e.g., `#3430, #3426`), treat each as an independent target. +2. **Sub-Agent Mandate**: To prevent polluting the main conversation context, **you MUST ALWAYS spawn a sub-agent** to perform the actual triage. + - **Parallelization**: When multiple IDs are provided, spawn one sub-agent per issue in parallel to improve efficiency. + - Do NOT run `glab` commands or read the label cache directly in the main context. + - In the `task` parameter for the sub-agent, provide the target `<issue-id>` and explicitly instruct the sub-agent to follow the "Triage Workflow" and "Asset Management" rules defined below. + - Wait for the sub-agent to finish and simply report its summary to the user. + +## Goal +Automate the initial triage of new bug reports and feature requests in the QEMU GitLab repository. + +## Prerequisites +- `glab` CLI tool installed. +- An authentication method: Use `glab auth status` to check. You can pass a token by using `env GITLAB_TOKEN=<token>` in front of commands if you have it. +- Target Repo: Use `-R qemu-project/qemu` if not in a clone, or ensure the remote is set correctly. + +## Asset Management (Label Cache) +This skill uses a cached list of labels to avoid unnecessary API calls and ensure consistent labeling. +- **Cache Location:** `assets/labels.txt` relative to this skill. +- **Updating the Cache:** If the user asks to update the labels, or if you suspect a label is missing, run the provided script: + ```bash + cd .agents/skills/qemu-issue-triage/scripts && ./update_labels.sh + ``` +- **Using the Cache:** Before applying labels, ALWAYS read `assets/labels.txt` (or use `grep` on it) to review the available labels and their descriptions. This ensures you use the exact spelling and understand the intent behind the label (e.g., `kind::Bug`, `Storage`). Do NOT guess label names. + +## Triage Workflow + +### 1. Information Gathering +Fetch the issue details: +```bash +glab issue view <issue-id> -R qemu-project/qemu --comments +``` + +### 2. Evaluate Completeness +Analyze the issue against the bug template requirements: +- **Host Arch/OS**: Is the host environment specified? +- **Guest Arch/OS**: Is the guest environment specified? +- **QEMU Version**: Is the version mentioned? +- **Reproduction Steps**: Are there clear steps to reproduce? +- **Expected vs Actual**: Is the bug clearly described? + +**Workflow::Needs Info Triggers (CRITICAL):** +Request more information and apply the `workflow::Needs Info` label if any of the following are true: +- **Missing Command Line**: The full QEMU command line (`qemu-system-* ...`) used to reproduce the issue is missing. +- **Old QEMU Version**: The reported version is older than the last two major releases. Ask the reporter to re-test with the current upstream master or the latest stable release. +- **Distro Version**: The version string suggests a downstream/distro-specific package (e.g., contains suffixes like `.el9`, `.fc40`, `-ubuntu`, or long strings like `7.2.0-14.sc05...`). Ask the reporter to reproduce the issue using a clean build from the current upstream source to rule out distro-specific patches. + +**Actions:** +- If information is missing based on the triggers above, add the `workflow::Needs Info` label and post a polite comment asking for the specific missing details. (Remember to sign the comment as "Issue Agent Bot on behalf of the user" per the Guidelines). +- If the issue is well-defined and uses a recent upstream version, proceed to categorization. + +### 3. Categorization & Labelling +Apply labels based on the issue content. **Crucially, consult `assets/labels.txt` to find the exact matching labels for the categories below.** + +#### Scoped Labels (Prefix::Label) +QEMU uses scoped labels (e.g., `kind::Bug`, `workflow::Triaged`) to group related categories. +- **Prefixes:** Common prefixes include `kind::`, `workflow::`, `Closed::`, `Audit Tooling::`, `GUI::`. +- **Constraint:** Only one label from that scope can be active at a time. Applying a new scoped label of the same type (e.g `workflow::`) will remove the other. + +#### Kinds +- `kind::Bug`: For unexpected behavior. +- `kind::Feature Request`: For new functionality. +- `kind::Task`: For research, investigations, and miscellaneous issues. + +#### Targets (target: *) and Hosts (host: *) +Detect the guest architecture (`target: *`) or host environment (`host: *`). +**IMPORTANT:** Be conservative when applying `target:` and `host:` labels. Many bugs (e.g., in generic devices like USB, PCI, or block controllers) apply to ANY guest that includes the device. The reproducer (like a `qtest` invocation) might just use a convenient target (e.g., `i386`) as an example. ONLY apply `target:` or `host:` labels if the bug is strictly architecture- or host-dependent (e.g., a bug in ARM CPU emulation, or a macOS-specific build failure). + +#### Accelerators (accel: *) +Detect the accelerator mentioned (e.g., `accel: KVM`, `accel: TCG`, `accel: HVF`). + +#### Subsystems +Identify the relevant subsystem (e.g., `Storage`, `Networking`, `device:virtio`, `Migration`). + +#### Testcases +- If the issue provides a minimal C program, a shell script, or a specific disk image to reproduce the bug, apply the `TestCase` label. + +#### Patches and Fixes +- If the issue description or comments contain a link to a patch on the mailing list (e.g., `lore.kernel.org`, `patchew.org`), or explicitly mention that a patch/fix has been submitted, apply the `workflow::Patch available` label. + +#### Workflow Management +The `workflow::` labels track the lifecycle of an issue. +- **Single Workflow Label:** An issue can only have one `workflow::` label at a time. +- **Transitioning:** Adding a new workflow label will automatically remove the old one (e.g., adding `workflow::Patch available` will remove `workflow::Triaged`). + +#### Other comments +- If other developers have commented see if those comments imply additional tags should be applied. + +### 4. Updating the Issue +Apply the labels and optionally assign a priority if clear. +**Transitioning Workflow Example:** +```bash +glab issue update <issue-id> -R qemu-project/qemu --label "workflow::Triaged,kind::Bug" +``` + +## Guidelines +- Be polite and professional in comments. +- **IMPORTANT:** Any comments added to the issue MUST include the phrase: "Issue Agent Bot on behalf of the user" (e.g., as a sign-off at the end of the message). +- Avoid commenting unless additional information is needed, specifically a comment to acknowledge the report is superfluous. +- Use `workflow::Triaged` once categorization is complete. +- Avoid assigning issues to specific people unless they are explicitly mentioned or are the known maintainer for a very specific subsystem. +- Use the `scripts/get_maintainer.pl` logic (via file paths mentioned in the issue) to identify potential subsystems. diff --git a/.agents/skills/qemu-issue-triage/assets/labels.txt b/.agents/skills/qemu-issue-triage/assets/labels.txt new file mode 100644 index 00000000000..12d3cc7e6f0 --- /dev/null +++ b/.agents/skills/qemu-issue-triage/assets/labels.txt @@ -0,0 +1,133 @@ +ACPI Power Management related (ACPI / SMBIOS / HEST / GHES) +Audio Audio devices; both backend (host audio) and frontend (guest audio) +Audit Tooling::AI For bugs found with AI assisted tools such as Mythos and other similar things +Audit Tooling::Fuzzer Issues found via fuzzing. For security issues, please consult https://www.qemu.org/contribute/security-process/ +Audit Tooling::Other Some unknown/unclassified type of audit tooling. +Audit Tooling::Sanitizer For issues found using sanitizers such as asan, lsan and tsan +Audit Tooling::Static Analysis Static analysis such as Coverity or more modern compilers. +Bite Sized Candidates for first contributions; see also https://wiki.qemu.org/Contribute/BiteSizedTasks +Build System configure, make, Meson, ninja, gcc, clang, ccache, etc. +CI Continuous Integration; gitlab, patchew, etc. +CLI Command Line Interface +Chardev Character device backends and related issues +Closed::Duplicate There is already another ticket that is about the same issue +Closed::Fixed The issue was fixed. Yay! +Closed::Invalid This issue was not really a problem +Closed::NotOurBug This is a bug, but not a bug in QEMU. Please report it and/or include a link to the report. +Closed::NotReproducible This issue was not reproducible +Closed::UnbackedFeature Nobody was willing to work on this feature request +Closed::WontFix The issue was acknowledged, but the fix would be too complex, too expensive, or would introduce other problems. +Cryptography +Documentation Sphinx documentation, man pages, the wiki, --help output, etc. +GDB Issues relating to using GDB via the gdbstub +GUI Graphical User Interface (gtk, SDL, curses, VNC, spice, ...) +GUI::Cocoa Cocoa +GUI::DBus DBus +GUI::GTK GTK +GUI::SDL SDL +GUI::SPICE SPICE +GUI::VNC +Guest Agent Issues related to the qemu-guest-agent binary. https://wiki.qemu.org/Features/GuestAgent +Hard +Launchpad Issues migrated from Launchpad +Migration +Modules +Networking +Python Python library issues (./python/) +QAPI/QMP QEMU API / QEMU Machine Protocol, HMP and CLI, etc. +QOM QEMU Object Model +Regression This is a regression from previously working behaviour +Security This is a security issue - see https://www.qemu.org/docs/master/system/security.html for guidance +Semihosting Semihosting calls provide a simple ABI for early bring-up of embedded devices, see https://www.qemu.org/docs/master/about/emulation.html#semihosting +Softfloat QEMU's FPU emulation code (TCG only) +Stable::can't fix The bug was reported on a stable branch but the fix is too invasive for backporting +Stable::obsolete The bug was reported on a stable branch that is not maintained anymore +Stable::to backport The bug was reported on a stable branch and needs to be backported on the next release from the branch +Storage Block subsystem, Storage devices, etc. +TCG plugins Anything related to the TCG plugins feature +TestCase The report includes a testcase +Tests qtests, iotests, acceptance tests, VM tests, docker tests, and more. See https://www.qemu.org/docs/master/devel/testing/index.html +USB +VFIO +accel: HAX Intel's Hardware Accelerated Execution Manager (HAXM) +accel: HVF Apple Hypervisor Framework +accel: KVM Linux Kernel-based Virtual Machine +accel: TCG QEMU Tiny Code Generator +accel: WHPX Microsoft Windows Hypervisor Platform (WHPX) +accel: Xen Xen Hypervisor +block:9p The 9p network file system +block:NVMe +block:curl +block:nbd +block:nfs Issues related to the NFS backend +block:qcow2 +block:ssh +block:vmdk +bsd-user +device: PCI +device: TPM Trusted Platform Module (TPM) devices +device: iommu +device: mmio mmio and other directly emulated devices +device:graphics Issues relating to display device emulation, or rendering in general. See also "GUI". +device:input Keyboards, Mice, Touchscreens, HIDs, etc. +device:iommu IOMMU and SMMU +device:pflash Parallel NOR flashes emulation +device:sdmmc SD or (e)MMC cards emulation +device:virtio virtio-related issues. https://www.linux-kvm.org/page/Virtio +device:watchdog +efi EFI firmware related issues +flaky-ci For test cases that are flaky when run under our CI +gitlab +guest: AIX +guest: BSD Guest OS is BSD (NetBSD/FreeBSD/OpenBSD/etc) +guest: Linux Guest OS is Linux/Linux-based +guest: Windows Microsoft Windows guest +guest: macOS Apple macOS / Darwin as guest OS +guest: os2 +host: aarch64 Bugs reproducible on AArch64 hosts +host: arm Bugs reproducible on ARM hosts +host: loongarch64 Bugs reproducible on LoongArch64 hosts. +host: mips Bugs reproducible on MIPS hosts +host: ppc Bugs reproducible on Power hosts +host: riscv Bugs reproducible on RISC-V hosts +host: s390 Bugs reproducible on s390 hosts +host: sparc64 Bugs specific to Sparc64 hosts +host: x86 Bugs reproducible on x86 hosts +host:32bit These are mostly TCG related bugs where we sometimes struggle with emulating larger guests, especially atomic and address space issues. +hostos: BSD FreeBSD, OpenBSD, NetBSD, and derivatives as host OSes +hostos: Linux Linux-based host operating systems (Fedora, RHEL/CentOS, Debian, Ubuntu, openSuSE et al) +hostos: Windows Microsoft Windows host OS +hostos: macOS Apple macOS / Darwin as a host OS +icount issues relating to icount, deterministic execution and record/replay functionality +kind::Bug Bug or defect in functionality. +kind::Feature Request Feature request or new functionality. +kind::Task Research, investigations, and miscellaneous issues. +libvfio-user +linux-user issues with QEMU's lightweight user-mode emulator +qemu-img +sysadmin Issues related to QEMU project infrastructure. You should probably also check on the projects IRC channel if the issue is urgent - https://www.qemu.org/support/ +target: alpha DEC Alpha [alpha] +target: arm Arm AArch32 or AArch64 [arm, aarch64] +target: avr Atmel AVR [avr] +target: hexagon Qualcomm Hexagon [hexagon] +target: hppa Hewlett-Packard Precision Architecture; HP/HP, PA-RISC [hppa] +target: i386 Intel/AMD x86 [i386, x86_64] +target: loongarch loongarch64 target architecture +target: m68k Motorola 68000 [m68k] +target: microblaze Xilinx MicroBlaze [microblaze, microblazeel] +target: mips MIPS [mips, mipsel, mips64, mips64el] +target: nios2 Altera Nios II [nios2] +target: openrisc OpenRISC [or1k] +target: ppc IBM Power Architecture, PowerPC [ppc, ppc64, ppc64le] +target: riscv RISC-V [riscv32, riscv64] +target: rx Renesas RX [rx] +target: s390x IBM Z, SystemZ, zSeries [s390x] +target: sh4 Renesas SuperH [sh4, sh4eb] +target: sparc Sun Microsystems SPARC [sparc, sparc64] +target: tricore Infineon TriCore [tricore] +target: xtensa Tensilica Xtensa [xtensa, xtensaeb] +workflow::Confirmed Bugs that have been confirmed and reproduced. +workflow::In Progress Someone is working on this issue. +workflow::Needs Info Issue has insufficient information to verify. +workflow::Patch available A patch is available +workflow::Triaged Issue has been triaged and given a topic label. diff --git a/.agents/skills/qemu-issue-triage/evals/evals.json b/.agents/skills/qemu-issue-triage/evals/evals.json new file mode 100644 index 00000000000..572c0723fd4 --- /dev/null +++ b/.agents/skills/qemu-issue-triage/evals/evals.json @@ -0,0 +1,18 @@ +# SPDX-License-Identifier: GPL-2.0-or-later +{ + "skill_name": "qemu-issue-triage", + "evals": [ + { + "id": 1, + "prompt": "Update the cached list of GitLab labels for the QEMU project.", + "expected_output": "The skill should fetch the latest labels from GitLab and save them to the skill's assets directory.", + "files": [] + }, + { + "id": 2, + "prompt": "Triage issue #1234. Assume the user provided clear steps but no subsystem label is obvious.", + "expected_output": "The skill should read the cached label list (not fetch from GitLab) and apply relevant kind/target labels.", + "files": [] + } + ] +} \ No newline at end of file diff --git a/.agents/skills/qemu-issue-triage/scripts/update_labels.sh b/.agents/skills/qemu-issue-triage/scripts/update_labels.sh new file mode 100755 index 00000000000..1605eab4de5 --- /dev/null +++ b/.agents/skills/qemu-issue-triage/scripts/update_labels.sh @@ -0,0 +1,16 @@ +#!/usr/bin/env bash +# SPDX-License-Identifier: GPL-2.0-or-later +set -e + +if [ -z "$GITLAB_TOKEN" ]; then + echo "GITLAB_TOKEN is not set. Attempting to fetch via pass..." + export GITLAB_TOKEN=$(pass gitlab-api) +fi + +echo "Fetching labels from qemu-project/qemu..." +# Fetch labels using API and format as "Name Description" +glab api /projects/qemu-project%2Fqemu/labels --paginate | \ + jq -r '.[] | [ .name, .description ] | @tsv' | \ + column -t -s $'\t' > ../assets/labels.txt + +echo "Labels cached in assets/labels.txt" diff --git a/AGENTS.md b/AGENTS.md index 6a91c39638e..97774176425 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -36,6 +36,7 @@ You should use the following specialized skills for common tasks: - `qemu-code-reviewer`: For pulling and applying patch series from mailing lists. - `qemu-mail-thread`: For analyzing and parsing mailing list threads. - `qemu-issue-helper`: For fetching and summarising issue details from the bug tracker. +- `qemu-issue-triage`: For helping triage issues in the bug tracker. ## Source Code Layout (see `docs/devel/codebase.rst`) - **`accel/`**: Hardware accelerators (KVM, TCG, HVF, Xen, etc.) and architecture-agnostic acceleration code. -- 2.47.3
