>I think anyone who has experience with deep dynamic testing knows they >need automation tools with custom configuration ability, the ability to >record workflow, a framework to create custom tests, etc.
Absolutely. But Arian there are differing deployment models. You don't just touch an application once in it's life and leave it, right ? You're doing architecture reviews, reviewing the functional requirement and RBACs, reviewing code, doing integrated security testing, doing a final validation (or as a friend once put it over drinks " the big giant pen-test"). For any of those activities, you need real live, experienced skilled testers. Once it goes live, however, you may very well have a SOC, NOC, or even "security" team who is tasked with the continual scanning and "monitoring" of their space who's goal is to touch everything - however lightly - at least once very x days. For this type of scenario where bulk scalability counts over quality - AND A QUALITY ASSESSMENT AND VALIDATION WAS ALREADY PERFORMED- I would suggest a scanner monkey may be appropriate. Of course you would NEVER want that to be your ONLY assessment or validation. Chris, SPI had a product called DevInspect that performed static and dynamic analysis as a single product, and was definitely around before Aug '07. Not saying it was red-hot, just saying it was there. I'd like to see NTO. Given the slower dev times of the larger companies and begrudgingly slow addition of core capabilities to them, I'm really hoping that some of the "smaller guys" end up growing and filling niches. For instance, I've heard that one smaller player crawls every bit as well as a major player, and *much* better than the other major player, but while costing considerably less than either. NTO reps, feel free to spam me (me, not the list). I will say this: Chris I'm completely with you in that I'm convinced that the majority of the market buying scanners is not doing so based on any objective empirical testing, but rather on "who found what" or what they "like". I'm even saddened to say that I recently saw a presentation by an organization tasked and paid to perform objective empirical analysis of scanners, that literally ranked them based on what they found, with absolutely no testing ground truth. I'm even more strongly convinced that the majority of those running these tools completely underestimate the expertise required to properly operate them and realize full potential from them. Given the complexity of testing software these days you still really need to know what you're doing to eak out of them what little value they hold. Even with realizing their full potential, however, there's still a lot of work to be done beyond a scan to perform anything resembling a complete assessment. Of course, a human assisted SaaS model has the potential to fill the gap, but from what I'm the majority of organizations using scanners like WI and AS in-house don't. Heck, even some really big name firms selling rather expensive fancily marketed assessments don't. Shame, really. -Matt. -----Original Message----- From: Chris Wysopal [mailto:cwyso...@veracode.com] Sent: Tuesday, August 04, 2009 8:54 PM To: Arian J. Evans; Matt Fisher Cc: Kenneth Van Wyk; Secure Coding Subject: RE: [SC-L] IBM Acquires Ounce Labs, Inc. I wouldn't say that NTO Spider is a "sort of" dynamic web scanner. It is a top tier scanner that can battle head to head on false negative rate with the big conglomerates' scanners: IBM AppScan and HP WebInspect. Larry Suto published an analysis a year ago, that certainly had some flaws (and was rightly criticized), but genuinely showed all three to be in the same league. I haven't seen a better head-to-head analysis conducted by anyone. A little bird whispered to me that we may see a new analysis by someone soon. As a group of security practitioners it is amazing to me that we don't have more quantifiable testing and tools/services are just dismissed with anecdotal data. I am glad NIST SATE '09 will soon be underway and, at least for static analysis tools, we will have unbiased independent testing. I am hoping for a big improvement over last year. I especially like the category they are using for some flaws found as "valid but insignificant". Clearly they are improving based on feedback from SATE '08. Veracode was the first company to offer static and dynamic (web) analysis, and we have been for 2 years (announced Aug 8, 2007). We deliver it as a service. If you have a .NET or Java web app, you would cannot find a comparable solution form a single vendor today. -Chris -----Original Message----- From: sc-l-boun...@securecoding.org [mailto:sc-l-boun...@securecoding.org] On Behalf Of Arian J. Evans Sent: Tuesday, July 28, 2009 1:41 PM To: Matt Fisher Cc: Kenneth Van Wyk; Secure Coding Subject: Re: [SC-L] IBM Acquires Ounce Labs, Inc. Right now, officially, I think that is about it. IBM, Veracode, and AoD (in Germany) claims they have this too. As Mattyson mentioned, Veracode only does static binary analysis (no source analysis). They offer "dynamic scanning" but I believe it is using NTO Spider IIRC which is a simplified scanner that targets unskilled users last I saw it. At one point I believe Veracode was in discussions with SPI to use WI, but since the Veracoders haunt this list I'll let them clarify what they use if they want. So IBM: soon. Veracode: sort-of. AoD: on paper And more to come in short order no doubt. I think we all knew this was coming sooner or later. Just a matter of "when". The big guys have a lot of bucks to throw at this problem if they want to, and pull off some really nice integrations. Be interesting to see what they do, and how useful the integrations really are to organizations. -- Arian Evans On Tue, Jul 28, 2009 at 9:29 AM, Matt Fisher<m...@piscis-security.com> wrote: > Pretty much. Hp /spi has integrations as well but I don't recall devinspect > ever being a big hit. Veracode does both as well as static binary but as > asaas model. Watchfire had a RAD integration as well iirc but it clearly must > not haved had the share ounce does. > > -----Original Message----- > From: Prasad Shenoy <prasad.she...@gmail.com> > Sent: July 28, 2009 12:22 PM > To: Kenneth Van Wyk <k...@krvw.com> > Cc: Secure Coding <SC-L@securecoding.org> > Subject: Re: [SC-L] IBM Acquires Ounce Labs, Inc. > > > Wow indeed. Does that makes IBM the only vendor to offer both Static > and Dynamic software security testing/analysis capabilities? > > Thanks & Regards, > Prasad N. Shenoy > > On Tue, Jul 28, 2009 at 10:19 AM, Kenneth Van Wyk<k...@krvw.com> wrote: >> Wow, big acquisition news in the static code analysis space announced today: >> >> http://news.prnewswire.com/DisplayReleaseContent.aspx?ACCT=104&STORY=/www/story/07-28-2009/0005067166&EDATE= >> >> >> Cheers, >> >> Ken >> >> ----- >> Kenneth R. van Wyk >> KRvW Associates, LLC >> http://www.KRvW.com >> >> (This email is digitally signed with a free x.509 certificate from CAcert. >> If you're unable to verify the signature, try getting their root CA >> certificate at http://www.cacert.org -- for free.) >> >> >> >> >> >> >> _______________________________________________ >> Secure Coding mailing list (SC-L) SC-L@securecoding.org >> List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l >> List charter available at - http://www.securecoding.org/list/charter.php >> SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) >> as a free, non-commercial service to the software security community. >> _______________________________________________ >> >> > _______________________________________________ > Secure Coding mailing list (SC-L) SC-L@securecoding.org > List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l > List charter available at - http://www.securecoding.org/list/charter.php > SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) > as a free, non-commercial service to the software security community. > _______________________________________________ > > _______________________________________________ > Secure Coding mailing list (SC-L) SC-L@securecoding.org > List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l > List charter available at - http://www.securecoding.org/list/charter.php > SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) > as a free, non-commercial service to the software security community. > _______________________________________________ > _______________________________________________ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. _______________________________________________ _______________________________________________ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. _______________________________________________
