[SC-L] Re: Comparing Scanning Tools
Hi Jerry, as one of the creators of the tool you evaluated, I have to admit I have the urge to comment on your message one line at a time and explain each way in which the presentation you attended did not adequately explain what Fortify does or how we do it. But I don't think the rest of the people on this list would find that to be a very interesting posting, so instead I'm going to try to stick to general comments about a few of the subjects you brought up. False positives: Nobody likes dealing with a pile of false positives, and we work hard to reduce false positives without giving up potentially exploitable vulnerabilities. In some sense, this is where security tools get the raw end of the deal. If you're performing static analysis in order to find general quality problems, you can get away with dropping a potential issue on the floor as soon as you get a hint that your analysis might be off. You can't do that if you are really focused on security. To make matters worse for security tools, when a quality-focused tool can detect just some small subset of some security issue, the create labels it a quality and security tool. Ugh. This rarely flies with a security team, but sometimes it works on non-security folks. Compounding the problem is that, when the static analysis tool does point you at an exploitable vulnerability, it's often not a very memorable occasion. It's just a little goof-up in the code, and often the problem is obvious once the tool points it out. So you fix it, and life goes on. If you aren't acutely aware of how problematic those little goof-ups can be once some researcher announces one of them, it can almost seem like a non-event. All of this can make the hour you spent going through reams of uninteresting results seem more important than the 5 minutes you spent solving what could have become a major problem, even though exactly the opposite is true. Suppression: A suppression system that relies on line numbers wouldn't work very well. When it comes to suppression, the biggest choice you've got to make is whether or not you're going to rely on code annotation. Code annotation can work well if you're reviewing your own code, but if you're reviewing someone else's code and you can't just go adding annotation goo wherever you like, you can't use it, at least not exclusively. Instead, the suppression system needs to be able to match up the salient features of the suppressed issue against the code it is now evaluating. Salient features should include factors like the names of variables and functions, the path or paths required to activate the problem, etc. Customization: Of course the more knowledge you provide the tool, the better a job it can do at telling you things you'd like to know. But in the great majority of cases that I've seen, little or no customization is required in order to derive benefit from any of the commercial static analysis tools I've seen. In the most successful static analysis deployments, the customization process never ends--people keep coming up with additional properties they'd like to check. The static analysis tool becomes a way to share standards and best practices. Regards, Brian ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php
[SC-L] RE: Comparing Scanning Tools
Title: RE: Comparing Scanning Tools McGovern, James F wrote: I have yet to find a large enterprise that has made a significant investment in such tools. Ill give you pointers to two. Theyre two of the three largest software companies in the world. http://news.com.com/2100-1002_3-5220488.html http://news.zdnet.com/2100-3513_22-6002747.html Brian ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php
RE: [SC-L] RE: Comparing Scanning Tools
Title: Re: [SC-L] RE: Comparing Scanning Tools I think I should have been more specific in my first post. I should have phrased it as I have yet to find a large enterprise whose primary business isn't software or technology that has made a significant investment in such tools. Likewise, a lot of large enteprrises are shifting away from building inhouse to either outsourcing and/or buying which means that secure coding practices should also be enforced via procurement agreements. Has anyone here ran across contract clauses that assist in this regard? -Original Message-From: Gunnar Peterson [mailto:[EMAIL PROTECTED]Sent: Friday, June 09, 2006 8:48 AMTo: Brian Chess; Secure Mailing List; McGovern, James F (HTSC, IT)Subject: Re: [SC-L] RE: Comparing Scanning ToolsRight, because their customers (are starting to) demand more secure code from their technology. In the enterprise space the financial, insurance, healthcare companies who routinely lose their customers data and provide their customers with vulnerability-laden apps have not yet seen the same amount of customer demand for this, but 84 million public lost records later ( http://www.privacyrights.org/ar/ChronDataBreaches.htm) this may begin to change.-gpOn 6/9/06 1:45 AM, "Brian Chess" [EMAIL PROTECTED] wrote: McGovern, James F wrote: I have yet to find a large enterprise that has made a significant investment in such tools. Ill give you pointers to two. Theyre two of the three largest software companies in the world.http://news.com.com/2100-1002_3-5220488.htmlhttp://news.zdnet.com/2100-3513_22-6002747.htmlBrian ___Secure Coding mailing list (SC-L)SC-L@securecoding.orgList information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-lList charter available at - http://www.securecoding.org/list/charter.php * This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies. * ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php
RE: [SC-L] RE: Comparing Scanning Tools
Title: Re: [SC-L] RE: Comparing Scanning Tools At the RSA Conference in February, I went to a reception hosted by a group called "Secure Software Forum"(not to be confused with the company Secure Software Inc, which offers a product competitive to Fortify). They had a panel session where representatives from a couple of companies not in the software/technology business claimed that they're making contractual requirements in this area (i.e., that vendors are required to assert as part of the contract what measures they use to assure their code). So I guess there's proof by construction that companies other than Microsoft Oracle care. Having said that, it's completely at odds compared to what I see working for an ISV of a non-security product. That is, I almost never have prospects/customers ask me what we do to assure our software. If it happened more often, I'd be able to get more budget to do the analysis that I think all vendors should do:-( --Jeremy P.S. Since Brian provided a link to a press release about Oracle using Fortify, I'll offer a link about a financial services company using Secure Software: http://www.securesoftware.com/news/releases/20050725.html From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of McGovern, James F (HTSC, IT)Sent: Friday, June 09, 2006 12:10 PMTo: Secure Mailing ListSubject: RE: [SC-L] RE: Comparing Scanning Tools I think I should have been more specific in my first post. I should have phrased it as I have yet to find a large enterprise whose primary business isn't software or technology that has made a significant investment in such tools. Likewise, a lot of large enteprrises are shifting away from building inhouse to either outsourcing and/or buying which means that secure coding practices should also be enforced via procurement agreements. Has anyone here ran across contract clauses that assist in this regard? -Original Message-From: Gunnar Peterson [mailto:[EMAIL PROTECTED]Sent: Friday, June 09, 2006 8:48 AMTo: Brian Chess; Secure Mailing List; McGovern, James F (HTSC, IT)Subject: Re: [SC-L] RE: Comparing Scanning ToolsRight, because their customers (are starting to) demand more secure code from their technology. In the enterprise space the financial, insurance, healthcare companies who routinely lose their customers data and provide their customers with vulnerability-laden apps have not yet seen the same amount of customer demand for this, but 84 million public lost records later ( http://www.privacyrights.org/ar/ChronDataBreaches.htm) this may begin to change.-gpOn 6/9/06 1:45 AM, "Brian Chess" [EMAIL PROTECTED] wrote: McGovern, James F wrote: I have yet to find a large enterprise that has made a significant investment in such tools. Ill give you pointers to two. Theyre two of the three largest software companies in the world.http://news.com.com/2100-1002_3-5220488.htmlhttp://news.zdnet.com/2100-3513_22-6002747.htmlBrian ___Secure Coding mailing list (SC-L)SC-L@securecoding.orgList information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-lList charter available at - http://www.securecoding.org/list/charter.php*This communication, including attachments, isfor the exclusive use of addressee and may contain proprietary,confidential and/or privileged information. If you are not the intendedrecipient, any use, copying, disclosure, dissemination or distribution isstrictly prohibited. If you are not the intended recipient, please notifythe sender immediately by return e-mail, delete this communication anddestroy all copies.* ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php
RE: [SC-L] RE: Comparing Scanning Tools
At 2:32 PM -0400 6/9/06, Jeremy Epstein wrote: Having said that, it's completely at odds compared to what I see working for an ISV of a non-security product. That is, I almost never have prospects/customers ask me what we do to assure our software. I don't even get those questions for our security product ! -- Larry Kilgallen LJK Software ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php
