Hi Thomas,

On Sat, 25 Apr 2026 11:06:17 +0200 Thomas Koenig wrote:

> I also do not understand how this helps with what you write below:
> 
> > Over time, this helped a lot developers here to build an effective
> > insight about the issues that might pass unnoticed.  

git bisect is our friend.

We routinely analyze the git history during nasty bug resolution and
security incident analysis. To be fair, this is not a new practice
introduced with LLM adoption, but something we were used to well
before their advent. The goal is to learn from previous errors and
improve. And, sometimes, to double check that what is reported as a bug,
was not a years-old feature-request of the customer reporting it. :-D


Anyway, such practice is particularly useful with LLMs.

Once you see that a few bugs were introduced by overlooked LLM output,
you start seeing patterns about reviewers' blind spots and LLMs
decompression artifacts. You learn where to look, so to say...


A particularly nasty bug I personally faced was related to two checks
where equality was improperly included (<= instead of < and >= instead
of >, in two different classes) causing a very hard to reproduce error
in production.
After the usual analysis, we realized the bug was due to a subtle
deviation from the prompt that slipped under the nose of two expert
reviewers. Pretty expensive issue for the company (and the customer),
but really formative too.


Note that this is nothing new: if after a few reviews you realize
a junior dev is unable to properly write loop exit conditions, 
you develop an insight about his blind spots.

It's mostly the same.


Giacomo

Reply via email to