Hi Simon (Jakobi) - > On 07/25/2026 1:40 PM CEST Simon Jakobi <[email protected]> wrote:
> > > > If you are reviewing an MR, and any part(s) of your review are > > > > generated by an LLM, you should use essentially the same guidelines as > > > > above. That is, you must take responsibility for every line of your > > > > review (P2); > > > > > > So I think that it would be worth experimenting with LLM-generated > > > reviews, in order to reduce the load on human reviewers and in order > > > to catch more bugs and performance issues before they make it into the > > > code base. But if we require that the person who prompts the review > > > and reports it on the MR has to take full responsibility for that > > > review, I think that such an experiment is unlikely to stand a chance. > > > In particular, the person providing the review would then probably be > > > required to understand the generated review, and possibly the MR too. > > > As a consequence we would be less likely to get LLM-generated reviews > > > on the more complex MRs where they could be particularly helpful. > > > > > > I'm not sure if there have been any problems with LLM-generated > > > reviews so far? My suggestion would be to drop the responsibility > > > requirement for clearly marked LLM-generated MR reviews. > > > > Am I reading you correctly that your suggestion is to have code reviews > > where no human takes full responsibility for the review? > > Yes. I think this could be an effective way to reduce the workload on > human reviewers and to reduce the number of bugs and perf issues > introduced into GHC. > > My intention is not to replace human review, but to filter out some of > the problems with a code contribution before a human reviewer spends > their time on it. I was initially horrified about this idea (and assumed I'd misunderstood you), but after readng more I think it's something I could get behind if done properly. There's apparently a roiling conflict on the LKML about some of the details (is it requiured, does the contributor or the reviewer need to read the LLM output, who if anyone is burdened with the "non-negligible quantity of nonsense", etc.), but I have confidence we could come to some kind of midpoint where we find bugs and don't burden people with too much slop. My personal leaning would be to only have a tool that a reviewer could optionally use as a first check, and if the LLM claims to find a bug the reviewer must take time to understand and take responsibility for the claim of the bug's existence, but this probably merits further discussion. Cheers, Tom _______________________________________________ ghc-devs mailing list -- [email protected] To unsubscribe send an email to [email protected]
