Am Mo., 27. Juli 2026 um 22:40 Uhr schrieb Zubin Duggal <[email protected]>: > > >> 3. Prompt injection. Gitlab comments are an untrusted > >> input source and LLMs fundamentally cannot discriminate > >> between instructions and data. You can imagine prompt > >> injection attacks where hidden text in a gitlab comment, > >> attached bug reproducer, link etc. gets the model to do > >> various destructive things, like spamming the gitlab, > >> editing/deleting your prior comments, mining bitcoin on the > >> CI runners, attempting to inject malicious code into the > >> compiler, testcases, reproducers etc. > > > >I don't quite understand how this is connected to my offer. I may have > >to apply some safeguards in the review but it's still running on _my_ > >system. Regarding the "untrusted input source": accounts already need > >to be verified. Recent models are also increasingly resistant to > >prompt injection. > > I would advise against giving any LLM system "write access" > to the gitlab, where it can autonoumously post things, push > commits, trigger pipelines and so on without a "human in the > loop".
I agree, and I the service I'm offering doesn't require any of this. > > Recent models are also increasingly resistant to prompt injection. > > I find this to be untrue in my experience. LLMs process an > undifferentiated input stream, and while the harnessnes try > to impose some structure on it, it comes through things > approximating xml tags, json etc. However, tool output can > produce these "structured tags" just as easily and this gets > injected into the context stream with no escaping/validation > etc so the model has no chance of differentiating between > "legitimate" instructions like `<system-reminders>` etc > that actually come from the harness and those that come > from running arbitrary commands etc. I have a few proof of > concepts of this I can show you off-thread, you don't even > have to be too clever about it. I would be interested in running these against Claude Opus 5 which is claimed to be quite resistant. _______________________________________________ ghc-devs mailing list -- [email protected] To unsubscribe send an email to [email protected]
