At least as secure as Vulnerability Assessment Assurance Level P; or Q or R. Well, that's what I think we should be able to say. What we need is an open standard, that has been agreed upon by recognized experts, against which the absence of software security vulnerability can be measured - something which improves upon the failings of the Common Criteria.
What about a completely different approach, as chosen by the Sardonix project? Keep track of who has tested a particular product and what they have found. Keep track of the ability of testers to find things and the number of things that are missed. Combine these metrics into some level of assurance and some security rating....
"5 very good security reviewers have done extensive testing of this product and found a small number of vulnerabilities."
"2 reviewers made a cursory pass over the code and identified a few issues"
"100 reviewers found many bugs in this product over the last 12 mos, and the number of vulns seems to be coming down very slowly with each new revision"
These sort of statements can be made more formal, and each carries a lot of useful information about security and confidence. Of course its only as good as participation. I'm not sure the level of information sharing required to make this really work is present in the security community.
Tim Newsham http://www.lava.net/~newsham/ _______________________________________________ Full-Disclosure - We believe in it. Charter: http://lists.grok.org.uk/full-disclosure-charter.html Hosted and sponsored by Secunia - http://secunia.com/
