Hello Wehrwolfmann,
Thanks for the review. I reproduced successfully five of the six from your descriptions and code, and the sixth from an equivalent diff. I'm working on fixes. On "local": you're right, the docs overclaim. The tool never touches a source the package declares, but a first run fetches AUR metadata and the signed baseline. That wording is being corrected. On layer 6: no quarrel from me. What runs as root includes .install scriptlets and committed patches, so the tree-reading path should be the default. That's the same root cause as item 2. Yes please to commands, samples and logs for f15a54a1/9bb5f88c and the neovim-git runs. It's unreleased as yet; I'll ping you when a build is ready to retest. How would you like to be credited? As message was bounced by Rspamd, im replacing the URLs. > Emiliano Gandini Outeda > https://emiliano-go.com > [email protected] On Sunday, September 20th, 2026 at 8:17 AM, Alexander Berg <[email protected]> wrote: > Hello Emiliano, > > I spent a day running TrustSight 0.16.0 (6b91636) against real AUR history and > against diffs I built myself, on an Arch-based system, because layer 6 is > where > I keep landing too. I tried to test the claims rather than read them. Two hold > up exactly as written; the rest of this is what I would fix. > > Confirmed by measurement: > > - It does not execute the recipe. Marker lines placed at top level, inside > $(...), and in prepare/build/package left no trace; strace shows exactly two > execve per run, the CLI and the sandboxed tokenizer. The tool itself says > that a command substitution's result is not in the analysed text. > - It never contacts what the recipe declares. With source= pointing at > 10.255.255.1 and .invalid hosts there is no connect() to any of them. To be > precise, though: on a fresh HOME the run opens 28 connections of its own to > github [dot] com and aur [dot] archlinux [dot] org for the baseline and the > metadata snapshot. > "Local" is true about the package, not about the process. > - Coverage gaps are announced, not hidden: "Not vetted: the repository file > manifest was unavailable, so only the PKGBUILD was examined". That default is > why I trust the numbers it does print. > > What I would fix, worst first: > > 1. The verdict sentence and the rules disagree. verdict.py:147 and :169 > prepend the literal "Version bump. " with no check that pkgver changed, and > reporting.py:83 calls fallback_verdict unconditionally. On a diff where > pkgver stays 1.2.3 and source moves from github [dot] com to an IP address, > the > verdict reads "Version bump. modified PKGBUILD; added 1 source URL(s)" while > C003, Source URL Changed Without Version Bump, fires correctly underneath. > The engine knows; the one sentence a hurried user reads says the opposite. > > 2. qt5-styleplugins f15a54a1 comes back "No findings", 0/100. That commit adds > two committed .patch files, a third sha512sum, a second source entry, and > `for p in "$srcdir"/*.patch; do patch` in prepare(). The unread tree I > expected - it is flagged inconclusive. The PKGBUILD text I did not: the > neighbouring commit 9bb5f88c gets "checksums added or changed" for the same > kind of edit. > > 3. Unresolved variables turn routine packaging into HIGH findings. > gtk3-classic keeps its version in pkgver=${_gtkver}; old and new both read > "${_gtkver}", pkgver_changed is False, and C001 fires where C002 belongs > (structural.py:263): 4 of 5 real version updates in a 12-diff window, 6 of > 14 in a 30-diff window, C002 not once. > > 4. Repeat runs of the same command differ. neovim-git at 99d9d479 scored > 0/100, then 15/100 ("Maintainer first seen for this package"), then 0/100, > because inspected packages are written to the local database even without > --record. Determinism holds for the text path - 13 --json runs over one > input give a single sha256 across locales, with and without TERM - but not > across the novelty layer. Since reproducibility is part of your argument > against opaque automation, I would either write only under --record or say > this plainly in the docs. > > 5. The release baseline imports "0 known source URLs and 35587 maintainers", > so URL novelty has nothing to compare against and the only novelty signal a > new user gets is about maintainers. Credit where due: 0.16.0 already guards > D001 against the empty-corpus case at novelty.py:75, which is the right > instinct. > > 6. `review --all --limit 8 --depth 0` on a fresh HOME does not review: it > prints "Downloaded AUR metadata snapshot. Run again to review changes." and > exits after 13.3 s. The second run takes 2.1 s and works. First runs are > where people decide whether a tool is for them. > > Cost once warm: 0.030 s per diff, 50 diffs of gtk3-classic in 1.47 s. Cheap > enough to sit in front of every update, which is the whole point. > > On the layer model, my only quarrel is with the wording of layer 6. Users are > told to read the PKGBUILD, but what runs as root also includes .install > scriptlets and patches committed next to the recipe - exactly what item 2 > walks past. Make the tree-reading path the default and text-only the > exception, and the advice and the instrument would point at the same thing. > > One practical note: the unaffiliated AUR package is still there and tracking > you closely - trustsight 0.16.0-1, maintainer amiad, updated today. It builds > from /archive/refs/tags/ rather than the release tarball your own > packaging/aur/PKGBUILD uses; both pin sha256sums and neither is signed, so the > difference is which bytes GitHub happens to generate. > > Happy to share exact commands, samples and logs for any of the above. > > Wehrwolfmann > > On Sun, Sep 20, 2026 12:23 AM, Emiliano Gandini Outeda > <[email protected]> wrote: > > > Hello, > > > > I've been an Arch user for a couple of years now, and using the AUR since > > day 1. As we all know, in the past months the AUR as been compromised a few > > times, and though I see things are changing (new AURweb, better security > > around orphaned packages, etc), I think we might be missing a centralized > > definition of the different layers, and who is in control of the bytes > > there. > > > > > > 1. Upstream project: upstream devs > > > > 2. AUR platform (aurweb, git, SSH, accounts): DevOps > > 3. Maintainership transfer: AUR Users, gated by TUs and Devops > > 4. The PKGBUILD author: a stranger > > 5. Reactive triage: TUs and reporting users > > 6. Pre-install diff review: the end user > > 7. Build isolation: the user, opt-in (devtools, nspawn) > > 8. Install and runtime: root > > > > > > Full coverage of layer 1 is not achievable at upstream's scale. Layer 5 > > works but is slow by nature, and we are still finding compromised packages > > from the earlier waves. Layer 6 is the one I keep coming back to: it is the > > step users are told to own ("read the PKGBUILD"), it is correct, and it is > > the one most people skip - increasingly so, as the user base shifts away > > from developers due to Arch-based systems like CachyOS. > > > > I've seen the suggestion to involve private companies (I'm against that) > > and to rely on automated systems (I'm wary of that too). My concern is not > > automation as such, it is anything that turns a signal into a verdict and > > removes the decision from the user. A tool can help here without doing that. > > > > To that end I've been building TrustSight, a PKGBUILD update vetting tool: > > https://github.com/emiliano-go/trustsight/ > > > > > > > > It is deliberately an instrument, not a judge. It reads AUR PKGBUILD diffs, > > applies published detection rules, and reports both what it found and what > > it could not examine. It is deterministic, runs entirely locally, never > > builds or executes a PKGBUILD, never fetches a URL the package declares, > > and produces no verdict - the output is input to ahuman decision. A quiet > > result explicitly does not mean "safe". The goal is to make the mandatory > > diff review faster and to tell users exactly what to look for (unresolved > > variables, changed upstreams, and so on). > > > > > > That is also the difference from the automated systems: the ruleset is > > published and auditable, the result is reproducible and carries its own > > coverage gaps, and it never authorizes an update. A human still decides. > > The tool just makes that decision cheaper. Docs and the full threat model > > are at https://trustsight.org/ and in the repo. > > > > > > One thing worth flagging while I'm here: a copy of TrustSight was uploaded > > to the AUR by someone else, without my involvement. I'll file a deletion > > request. A tool that reviews AUR packages should not be > > distributed through the channel it reviews - it is circular, and in this > > case the upload is unaffiliated and therefore itself an unreviewed package. > > > > > > I'd welcome feedback, especially on the layer model above and on where a > > tool like this should stop. > > > > > > I invite you to read the security model at docs [dot] trustsight [dot] org > > [slash] security, or docs [dot] trustsight [dot] org. > > > > > > > Emiliano Gandini Outeda > > > https://emiliano-go.com > > > [email protected]
publickey - [email protected] - 0xF759D6D4.asc
Description: application/pgp-keys
signature.asc
Description: OpenPGP digital signature
