Hi!
It seems that there is a general story shining through what Kazu wrote:
1. We accidentally introduce lots of security issues into our software.
2. We don’t find these security issues but use LLMs to find them.
3. We cannot fix these security issues but use LLMs to fix them.
4. We don’t understand our codebases but use LLMs to make changes.
5. We cannot take responsibility for the changes made by LLMs but hope
that they don’t introduce new issues.
6. We include LLM-generated text into our documentation that we don’t
understand but let LLMs explain its meaning to us.
This seems like an absolutely worrying state of affairs. I wonder why we
got there:
• Did we care too little about design?
• Did we introduce too many features?
• Did we care too much about development speed and too litle about
quality?
• Did we pay too little attention to detail?
Apparently, the effects of multiple deficits have accumulated over the
years, and now we use LLMs to somehow handle the outcome. This may well
make our codebases even worse and at the same time lessen our abilities
to construct quality software.
All the best,
Wolfgang
Am Fr 23.10.2026 10:33 schrieb Kazu Yamamoto (山本和彦) via ghc-devs:
> Hi,
>
> Encouraged by Simon's understanding message, I would like to share my
> own perspective. I am one of those who have submitted merge requests
> containing code generated directly by Claude Code. Unfortunately, this
> action caused some people discomfort. It is natural for a wide range
> of opinions to emerge during a period of significant change, so I am
> not surprised that some hold views diametrically opposed to
> mine. Furthermore, it is certainly not my intention to attack anyone
> with this message. I intend to follow GHC's development policies, and
> if I have violated them, I will make the necessary corrections.
>
> I maintain code related to networking. Over the past few months, I
> have received numerous reports of vulnerabilities identified by
> AI. Initially, I would personally analyze and fix these issues, or
> carefully review and merge the resulting pull requests. However, the
> volume eventually exceeded my capacity.
>
> As a countermeasure, I decided to use AI to proactively address these
> vulnerabilities myself, and I began using Claude Code since September
> 15th. Claude Code uncovered a vast number of vulnerabilities, a
> revelation that truly shocked me. It became clear to me that the only
> viable strategy is to run faster than bad guys. Just yesterday, a
> vulnerability concerning HTTP/3 header compression was
> disclosed. However, Claude Code had already addressed it in the
> Haskell "http3" library three weeks prior. While there may be
> differing opinions on this, I am proud of what was achieved in this
> instance.
>
> https://github.com/kazu-yamamoto/http3/issues/54
>
> My current work involves developing a DNS server in Haskell. For a
> long time, I struggled with rare, intermittent bugs, specifically,
> workers stalling and the system exhausting its memory. Suspecting that
> these were not bugs in the DNS server itself, but rather issues within
> the GHC runtime, I utilized tools like Claude Code to analyze them.
>
> I have previously contributed to the GHC project, specifically
> regarding the IO Manager. In this recent process, Claude Code
> identified several bugs within the IO Manager. The migration of GHC
> development to gitlab raised the barrier to entry for contributors.
> However, Claude Code helped me understand the proper procedures,
> enabling me to submit merge requests. It was pointed out that my
> merge request description, commit messages, and code comments were
> overly verbose, so I rewrote them all. This was not an issue for me,
> as I am responsible for the IO Manager.
>
> Meanwhile, I have also been struggling with a bug in Haddock. A
> regression in Haddock has broken the structure of the documentation I
> manually wrote. I requested a fix multiple times, but as it remained
> unresolved, I used Claude Code to demonstrate how it could be fixed,
> though I later trimmed down the verbose explanations I had included.
>
> However, I would find it problematic to be held responsible for the
> code or comments within Haddock. I am not familiar with Haddock's
> internals, nor do I have the time to learn them. If I am expected to
> take responsibility for such areas, I will have to cease contributing
> to anything outside of the IO Manager, as it simply exceeds my
> capabilities.
>
> To date, I have never asked anyone to rewrite AI-generated text sent
> regarding my own software. I can simply have an AI summarize it, and I
> also fear that making such a request might discourage others from
> providing feedback altogether.
>
> As a native Japanese speaker, a language with a grammar entirely
> distinct from English, I find myself constantly grappling with the
> English used by others in this community. I rely on machine
> translation and dictionaries just to grasp the details. I am not even
> entirely confident that this very message has been translated
> accurately. Using AI to summarize content of issues/MRs for the
> purpose of understanding it is not significantly different from the
> methods previously used to understand English.
>
> From the perspective of someone who maintains network-related code,
> which is constantly exposed to external threats, the GHC community
> strikes me as being quite laid-back regarding the adoption of AI.
>
> --Kazu
>
> From: Simon Peyton Jones via ghc-devs <[email protected]>
> Subject: Re: Notes/Commit messages/Comments - A new AI discussion
>
> > Perhaps I can add my perspective.
> >
> > This thread is largely about LLM generated tickets. Mikolaj in
> > particular
> > has been posting a series of a couple of dozen such tickets -- so many
> > in
> > fact that I started collecting them on a tab of my personal WIP
> > spreadsheet. You can see it here
> > <https://docs.google.com/spreadsheets/d/1o05HVgAnsSjuLNFyDZ78DJPIirw664i7i48g6aHan5w/edit?usp=sharing>
> > (look at the Mikolaj tab). From this I have learned
> >
> > - All of them that I have investigated so far have proved genuine --
> > they expose genuine shortcomings in GHC.
> > - It's pretty overwhelming, even discouraging, to get so many in a
> > short
> > time. And yet, and yet.... I would rather know than not-know.
> > - The explanations are often long, but I zoom in on the reproducer,
> > and
> > only then refer back to the explanation. As Mikolaj says, these
> > reproducers are far, far, far better than the gigantic reproducers he
> > used
> > to offer. Provided the ticket has a short statement of what is
> > wrong, and
> > a reproducer, I don't mind if there is AI-generated goop to accompany
> > it.
> > Indeed, once I can reproduce it, the goop is often somewhat helpful.
> > - I say "somewhat" because once I have understood the problem, I can
> > often come up with a much crisper summary myself Here's a concrete
> > example
> > <https://gitlab.haskell.org/ghc/ghc/-/work_items/25526#note_705423>,
> > with my summary belo.
> > - It would be absurd to expect Mikolaj to write every word. He
> > doesn't
> > know enough about GHC's internals. If I insisted on that, he just
> > wouldn't
> > submit them, and that would be a shame.
> > - *The biggest shortcoming is this: I have no idea how important the
> > ticket is*. When all tickets were human generated, I knew that every
> > single one was somewhat important to some human. I don't want to
> > spend
> > time fixing bugs that no one cares about, and I'm 100% against
> > LLM's autonomously submitting tickets. Every ticket must have a
> > human
> > behind it. Meanwhile, I asked Mikolaj to prioritise his flood of
> > tickets,
> > which he has done as you'll see on the spreadsheet.
> > - One way in which the flood can feel exhausting is simply because it
> > exposes so many embarrassing shortcomings in the compiler on which I
> > have
> > lavished so much love and care. How can I have been so stupid in so
> > many
> > different ways? Why is GHC so over-complicated? It is humbling.
> > But in a
> > good way, I think.
> >
> > Finally, while tickets may have a lot of readers, they are ephemeral.
> > What
> > I *really *care about is the code and the Notes in the code. It will be
> > read for decades, whereas no one in 10 years time is going to be reading
> > today's tickets. (There are occasional exceptions.) For the code and
> > Notes I want every line to be human written, or so well studied, audited
> > and revised that it might as well be human written.
> >
> > TL;DR: for me, provided a ticket has a human author who actively cares
> > about, clear signposting and attribution, and a concrete reproducer, I'm
> > pretty relaxed about accompanying slop.
> >
> > Relatedly, let me add that I have also been requesting *LLM reviews* for
> > my
> > MRs, through Simon Jacobi's good offices, and I have been astonished how
> > good they are. They often identify genuine issues with my lovingly
> > crafted
> > MRs, and ones which both I and human reviewers have missed. At the time
> > it's a bit "Oh no, I wanted to land this and now I can't", but again it
> > must be better to know about the bugs than not to know. Again it's a
> > humbling experience.
> >
> > So thus far, I have had a pretty positive experience of LLM-generated
> > bug
> > reports and reviews. But as I say above, I am much, much more cautious
> > about contributions to GHC's code base and documentation.
> >
> > Simon
> >
> > On Thu, 22 Oct 2026 at 21:45, Mikolaj Konarski via ghc-devs <
> > [email protected]> wrote:
> >
> > > > "[E]very contribution must have an identifiable human author (or
> > > authors), who takes full responsibility for their entire
> > > contribution"
> > >
> > > This is precisely why I advocate to "embrace the slop", that is,
> > > treat LLM-generated content as raw data, as if the content was
> > > a log, a crash message, a debug session transcript.
> > > But let me start by describing my experience with LLMs and GHC
> > > so that my position can be better understood.
> > >
> > > I'm recently posting many GHC issues with LLM-generated
> > > data and, while I hope to over time improve the way I do it,
> > > I see that their quality is much better than when I I did
> > > not use LLM for two reasons:
> > >
> > > 1. Before, I was posting huge reproducers and bright minds
> > > were spending weeks minimizing them or sometimes they
> > > could not. Now LLMs miminize all my reproducers.
> > >
> > > 2. I have no clue about GHC internals, so I could only guess
> > > if my complaint is intended and/or known behaviour, which bugs
> > > are already reported in a way that I don't recognize, which defect
> > > is just a known difficult trade-off inherent in GHC heuristics
> > > biting me
> > > occasionally and so not worth a report if I don't have a body
> > > of statistical evidence to support a heuristic overhaul proposal.
> > > Today LLMs recognize each of these scenarios for me and I'm
> > > withdrawing half of my attempted defect reports.
> > >
> > > As a bonus, when I'm reading the slop to summarize it in a sentence
> > > or two at the issues' start, I'm learning GHC internals. I'm
> > > learning
> > > in particular from the often correct root cause analysis by LLMs
> > > and from the often incorrect fix proposals. This is one reason
> > > giving
> > > me hope my future GHC LLM-assisted contributions will be higher
> > > quality.
> > > Another is GHC devs' feedback. Yet another is, I hope, LLMs getting
> > > better.
> > >
> > > Back to embracing the slop: I can't "take full responsibility" for
> > > what
> > > LLM writes, because I'm clueless. It's the same as if I was posting
> > > a transcript of a clueless debug session --- I totally can't take
> > > responsibility for my working hypotheses. Neither, when I minimize
> > > a big performance regression, I can't take responsibility for
> > > the guess that the 3% slower minimized repo represents
> > > the same mechanism that slowed down my original big program
> > > by 30% (which is the only argument why it's worth looking at).
> > >
> > > What I can take full responsibility for is my user story
> > > and the dumb raw data. "This log is from commandline X".
> > > "This gdb output is from my program interrupted at Y".
> > > "This LLM output is from Z".
> > >
> > > I hope we can improve our understanding not only what
> > > kind of LLM-generated data is helpful, but also which kind of "Z"
> > > is most honest and most helpful to all people concerned.
> > > Is it mixing LLM-content with human-content? Certainly
> > > not for me, because I'm too clueless to correct an LLM
> > > and by mixing I'm diluting the attribution and introducing errors.
> > > Should Z be the prompt? A summary in my own often misguided
> > > words clearly separated from the slop (this is what I do currently)?
> > >
> > > All the best,
> > > Mikolaj
> > > _______________________________________________
> > > ghc-devs mailing list -- [email protected]
> > > To unsubscribe send an email to [email protected]
> > >
>
>
> _______________________________________________
> ghc-devs mailing list -- [email protected]
> To unsubscribe send an email to [email protected]
_______________________________________________
ghc-devs mailing list -- [email protected]
To unsubscribe send an email to [email protected]
