On Tue, 4 Aug 2009, Chris Wysopal wrote: > As a group of security practitioners it is amazing to me that we don't > have more quantifiable testing and tools/services are just dismissed > with anecdotal data. I am glad NIST SATE '09 will soon be underway and, > at least for static analysis tools, we will have unbiased independent > testing. I am hoping for a big improvement over last year. I especially > like the category they are using for some flaws found as "valid but > insignificant". Clearly they are improving based on feedback from SATE > '08.
By the way, I don't recall anybody mentioning this to SC-L before, but the SATE 2008 writeup and raw data are available: http://samate.nist.gov/index.php/SATE.html In the NIST pub we cover a lot of lessons learned, especially in my paper. >From the raw data you can see the complexities in doing this kind of large-scale comparison. In my opinion, our biggest limitation was not using live tools. - Steve _______________________________________________ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. _______________________________________________
