On 4/28/26 09:58, Jeremy Stanley wrote:
As I'm sure is the case for everyone, the projects I work in are under
a seemingly unending deluge of vulnerability reports from researchers
using LLMs to mine for security gold in our software. At the same
time, we see maintainers on our projects relying on LLM-oriented tools
to develop fixes for vulnerabilities and compose prose for advisories.
While I take a moment to catch my breath, this new Bizarro World we're
all living in has gotten me thinking about the risks of public LLM
services to embargoed vulnerability handling workflows and traditional
coordinated disclosure. The operators of these LLM services are known
to feed prompts and results back into their training data, presumably
making it faster and easier for the same information to be found later
by other users of the same service. Would keeping embargoes short help
to mitigate related risks of parallel rediscovery or outright
disclosure to other LLM users? It seems to me that there must be some
inherent lag in this process, but how much?
The biggest risk is parallel discovery. If an LLM can find a bug for a
whitehat, it can do the same for a blackhat.
I'm sorely tempted, both due to the increased volume and the risk of
premature disclosure, to just assume that any vulnerability reported
as a result of research using an LLM is trivially discoverable by
others, and give up trying to pretend there's any point to working it
under embargo.
You are correct here: you should assume that any LLM will give a
similar result to another person who asks a similar question. In other
words, LLM-discovered vulnerabilities should be considered already
publicly known.
Similarly, it makes sense to me that patch development and descriptive
prose shouldn't be produced with LLM assistance for any vulnerability
that is being worked under an embargo.
This is not a problem if, and only if, you only use local LLMs. Use of
cloud LLM services should be assumed to break an embargo---they would be
immensely valuable "watering hole" targets for blackhats if such use
became common.
-- Jacob
