On Mon, Jun 18, 2012 at 10:20:35AM -0700, Jon Callas wrote: > On Jun 18, 2012, at 5:26 AM, Matthew Green wrote: > > > The fact that something occurs routinely doesn't actually make it a good > > idea. I've seen stuff in FIPS 140 evaluations that makes my skin crawl. > > > > This is CRI, so I'm fairly confident nobody is cutting corners. But that > > doesn't mean the practice is a good one. > > I don't understand. > > A company makes a cryptographic widget that is inherently hard to > test or validate. They hire a respected outside firm to do a > review. What's wrong with that? I recommend that everyone do > that.
When the vendor of the product is paying for the review, _especially_ when the main point of the review is that it be publicly released, the incentives are all pointed away from looking too hard at the product. The vendor wants a good review to tout, and the reviewer wants to get paid (and wants repeat business). I have seen cases where a FIPS 140 review found serious issues, and when informed the vendor kicked and screamed and threatened to take their business elsewhere if the problem did not 'go away'. In the cases I am aware of, the vendor was told to suck it and fix their product, but I would not be so certain that there haven't been at least a few cases where the reviewer decided to let something slide. I would also imagine in some of these cases the reviewer lost business when the vendor moved to a more compliant (or simply less careful) FIPS evaluator for future reviews. I am not in any way suggesting that CRI would hide weaknesses or perform a lame review. However the incentives of the relationship do not favor a strong review, and thus the only reason I would place credence with it is my impression of the professionalism of the CRI staff. In contrast, consider a review by, say, a team of good grad students, where the incentive is very strongly to produce a publishable result and only mildly on making the vendor happy. Those incentives again are not perfect (what is), especially given how academic publishing works, but they are somewhat more aligned with the end users desire to have a product that is secure. > Un-reviewed crypto is a bane. Bad crypto with a rubber stamp review is perhaps worse because someone might believe the stamp means something. -Jack _______________________________________________ cryptography mailing list cryptography@randombit.net http://lists.randombit.net/mailman/listinfo/cryptography
