I'd really like to know who Clive Robinson is; he's brilliant, I've found myself thinking that multiple times while reading Schneier's blog.
Check out the paragraph on Faux entropy, it's... very interesting, Doctor. ==== Clive Robinson * October 14, 2013 5:11 PM https://www.schneier.com/blog/archives/2013/10/insecurities_in.html#c1908693 @ Adrian Ratnapala, *If there is a vulnerability (especially an academic sounding one like this) then I am not surprised it comes via the entropy estimation. Ted Tso claims(and I believe) that it's just a *cking hard thing to do* Actualy it's a bit more than "*cking hard thing to do" it can easily be shown to be in effect impossible to do by any resource limited entity, irrespective of what statistical tests you come up with (both diehard and dieharder are a bit of a joke these days as many fully determanistic systems pass them with little difficulty). Basicaly if you take any old garbage generator such as say a plain integer counter, you can then "whiten it" with a basic crypto function such as a block cipher or hash algorithm. The result will pass any of the "usual suspect" "not random" witness tests and thus appear to be sufficiently random for "crypto use". And as has been mentioned on this blog before hardware engineers that fail to meet "Diehard" with their basic TRNG source can make it meet diehard by simply whitening it with AES etc which may also be built into the chip (as is the case with Intel). Unless the chip manufacture provides you with access to the output of the raw source generator so you can test it, then to be honest I would not trust it as I would not be able to verify it functioning correctly. Intel have consistantly failed to provide access to the raw source therefore I for one am not going to bet my life on their generator nor would I bet the lives of my loved ones or others I know. The reason why "entropy estimation" is so hard is in part due to the definition of "random" and in part because we know no way to measure what qualifies as a "random sequence". Thus you could say the output of any entropy generator is the sum (or in some cases the product) of, 1, Real Entropy. 2, Faux Entropy. 3, Detectable determanism. 4, Detectable bias. What you are after is (1) the real entropy in your physical source left after you have removed (3) the determanism and (4) bias that you have been able to detect and remove. Unfortunatly that leaves a lot of (2) faux entropy which you cannot detect and remove for a very large number of reasons. Faux entropy consists of many things including the outputs of complex processes and chaotic processes inherant as side effects of the design process. One example of which is to have two free running oscillators one running at high speed into the D input of a D-type latch and the slower one going into the clock input of the D-type. Close in observation of the Q output of the latch shows both metastability and what appeares as random behaviour. However observation at a slower time base shows "bunching" patterns that corespond on the timebase to the lowest frequency difference of the two oscilators. And in fact if driven into a suitable counter circuit will produce a sinusoudal output count... Taking what is in effect an FFT of the random source output will show up this determanistic wave, only if it is within the FFT Window range... Thus great care should be practiced on estimating what part of such a signal is real or faux entropy. Which brings me onto your question of, *So the question is, how can we do better? Since I doubt entropy estimation is going to get much less unreliable* The simple answere is "nothing" unless we are prepared to use increasing levels of computation to run more "witness to non randomnes tests". What we can do on the practical side is run more "faux generators" against our "entropy pool" in a way that would make any determanistic process hidden by an adversary next to impossible to see. One such way is to take the output of the OS RNG through what is in effect a blind encryption algorithm. -- http://www.subspacefield.org/~travis/ Remediating... LIKE A BOSS
pgpjh2h6hns2c.pgp
Description: PGP signature
_______________________________________________ RNG mailing list [email protected] http://lists.bitrot.info/mailman/listinfo/rng
