This paper describes how to use open weight LLMs to hide text messages inside other messages of the same length. https://arxiv.org/abs/2510.20075
The technique is to use the LLM to rank the tokens of the hidden message by their probabilities, then generate an unrelated cover text using a secret prompt and the same list of token ranks. To read the hidden message you use the secret prompt to recover the list of token ranks and then generate the hidden message by omitting the prompt. This requires that the sender and receiver both have access to the same weights, a model small enough to run locally (they use Llama 3 8B), and know the secret prompt. One problem they note is that the cover text looks less plausible and has lower probability than natural text. The most interesting thing I found from the paper is that LLMs can guess the next token on the first try 40% of the time, similar to how Shannon found in 1950 that humans can guess the next character 80% of the time. In both cases, the actual probability is much higher than 40% or 80%, sometimes close to 100%. Shannon estimated the entropy of written English between 0.6 and 1.3 bits per character. The uncertainty comes from only knowing the ranking and not the true probabilities that led to them, but with LLMs we have both. I still don't know exactly because the figure on page 5 is hard to interpret, but I think it shows that random posts on Reddit have 40% as much entropy as random words. So it seems obvious to me that the cover text plausibility problem could be fixed by skipping or combining high probability tokens, but the authors didn't try it. More generally I find it difficult to think of a use case. We can already hide arbitrary encrypted messages inside images or video by twiddling the low bits without visibly changing the appearance, but the message has to be much smaller than the cover image. In any case, if you have to send a secret key to read the message, why not send the message directly? One use case is time stamping, proving that you knew something when the cover was posted publicly by later revealing the key. But I think that guessing a secret prompt would be more vulnerable to attack than conventional cryptography. -- Matt Mahoney, [email protected] ------------------------------------------ Artificial General Intelligence List: AGI Permalink: https://agi.topicbox.com/groups/agi/T90d65db8ef43a62b-M01258459e42aad392c60df38 Delivery options: https://agi.topicbox.com/groups/agi/subscription
