Hi Gary, No offense taken. :) Securing Web software is a plenty big enough challenge for me. 270+ million websites accessible to 2 billion people. And let's not even go into the hundreds of thousands of mobile apps, which are basically all mini webapps. After I'm done solving that problem I'll think about moving on to mainframes, operating systems, and databases. uhh, maybe not.
One thing in your note deserves further WebAppSec clarity: "You can't push dynamic testing very far back in the SDLC, because your code has to run before you can test it dynamically. For me, way back in the SDLC means architectural risk analysis or even security requirements analysis." Very true, no one appreciates this better than us. As you probably know the Agile approach to software development is winning out in the Web application world. Release early. Release fast. Release often. Push or die on Tuesday, that's the motto. This means the time distance between architectural risk analysis / security requirements analysis and production deployment is roughly 2 - 4 weeks (6 max). Whatever analysis necessary to perform must also be repeated on each iteration in case a change impacts the entire system. Maybe you do not agree? Then QA testing windows are shortened to 1 - 3 days regardless of the preferred method of software security testing (SAST or DAST). Collective then this all begs the question, what forms of software security checks does the enterprise have the time and resources to deploy. Regards, Jeremiah- On Feb 4, 2011, at 2:47 AM, Gary McGraw wrote: > hi arian, > > Glad you liked the article. > > I guess my brush was a bit too wide when it comes to dynamic testing. I > was really only referring to the Web application testing tools which in my > mind "hit the wall" for two reasons. Reason one is that they only work > over port80 and are designed to take advantage of the fact that HTTP is a > stateless protocol (with a few small caveats). IMPORTANT NOTE: lots of > software is not web software (sorry Jeremiah). Reason two has to do with > the canned nature of the tests. The generic tests in the black box Web > app testing tools are, well, generic. If your software falls prey to > those tests, it sucks. IMPORTANT NOTE: Lots of software does, in fact, > suck. > > As you probably know I call those tools "badness-ometers" and also > ***suggest that everyone buy and use one***. See this ancient post (and > associated informIT article) from 2007: > > http://www.cigital.com/justiceleague/2007/03/19/badness-ometers-are-good-do > -you-own-one/ > > Now, there are many other kinds of dynamic testing tools (think of any > kind of fault injection tool). I wrote a software engineering tome about > that way back in 1998 called Software Fault Injection > <http://tinyurl.com/4ao6twv>. And you are right that dynamic testing has > a place. However, short of fuzzing tools generally tied to a > grammar-based protocol and capture-replay tools there are not very many > dynamic testing tools that work for non-Web software. Why not? Because > genericizing is too hard, making the potential market for a particular > tool too small. > > Security testing plays a key role in the Touchpoints (my own and Cigital's > approach to SDLC integration) which are described in Software Security > <http://swsec.com>. Hoglund and I also describe some dynamic tools that > we screwed around with when writing Exploiting Software in 2004 > <http://www.exploitingsoftware.com/>. I am in complete agreement that > dynamic testing is important for software security. > > One quibble with your question. You can't push dynamic testing very far > back in the SDLC, because your code has to run before you can test it > dynamically. For me, way back in the SDLC means architectural risk > analysis or even security requirements analysis. > > Sorry for the multiple invocations of the way back machine! I must be > getting old. > > gem > > company www.cigital.com > podcast www.cigital.com/silverbullet > blog www.cigital.com/justiceleague > book www.swsec.com > > > > On 2/3/11 7:26 PM, "Arian J. Evans" <arian.ev...@anachronic.com> wrote: > >> Great article, Gary. Many of your comments about static technology >> challenges I have seen and verified first-hand, including >> multi-million dollar cost overruns. After some great dialogue with >> John Stevens, I suspect we have had similar experiences. >> >> I was just about to write a similar article at a higher level - about >> how the vast majority of enterprise customers I work with are actively >> moving security into the SDLC. The time has come, the event has >> tipped, and SDLC security is indeed mainstream. This is an exciting >> time to be in the industry. >> >> However - I was curious about your comments about dynamic tools >> "reaching their limit" or something like that, as customers move >> security efforts deeper into the SDLC. What does that mean? >> >> I see customers making extensive use of dynamic testing, and >> leveraging it deeper and deeper into the SDLC. Enterprises are >> aggressively rolling out and expanding dynamic testing earlier in the >> SDLC. Newer dynamic testing technologies help solve/reduce some of the >> key pain points that static technologies alone are causing them, as >> you so well illustrated.. >> . >> I am very interested in why you sound dismissive of these successful >> technologies? Your article makes it sound like they are hitting some >> invisible limit, when in fact hundreds of enterprises are expanding >> dynamic testing in the SDLC. And these are serious projects that run >> into the 7-figures. >> >> Any insight you can share would be appreciated! >> >> Great work identifying the general shift SDLC security is moving >> mainstream, >> >> --- >> Arian Evans >> Software Security Referee >> >> >> >> On Wed, Feb 2, 2011 at 6:48 AM, Gary McGraw <g...@cigital.com> wrote: >>> hi sc-l, >>> >>> John Steven and I recently collaborated on an article for informIT. >>> The article is called "Software [In]security: Comparing Apples, Oranges, >>> and Aardvarks (or, All Static Analysis Tools Are Not Created Equal)" and >>> is available here: >>> http://www.informit.com/articles/article.aspx?p=1680863 >>> >>> Now that static analysis tools like Fortify and Ounce are hitting the >>> mainstream there are many potential customers who want to compare them >>> and pick the best one. We explain why that's more difficult than it >>> sounds at first and what to watch out for as you begin to compare tools. >>> We did this in order to get out in front of "test suites" that purport >>> to work for tool comparison. If you wonder why such suites may not work >>> as advertised, read the article. >>> >>> Your feedback is welcome. >>> >>> gem >>> >>> company www.cigital.com >>> podcast www.cigital.com/silverbullet >>> blog www.cigital.com/justiceleague >>> book www.swsec.com >>> >>> _______________________________________________ >>> Secure Coding mailing list (SC-L) SC-L@securecoding.org >>> List information, subscriptions, etc - >>> http://krvw.com/mailman/listinfo/sc-l >>> List charter available at - http://www.securecoding.org/list/charter.php >>> SC-L is hosted and moderated by KRvW Associates, LLC >>> (http://www.KRvW.com) >>> as a free, non-commercial service to the software security community. >>> Follow KRvW Associates on Twitter at: http://twitter.com/KRvW_Associates >>> _______________________________________________ >>> > > > _______________________________________________ > Secure Coding mailing list (SC-L) SC-L@securecoding.org > List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l > List charter available at - http://www.securecoding.org/list/charter.php > SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) > as a free, non-commercial service to the software security community. > Follow KRvW Associates on Twitter at: http://twitter.com/KRvW_Associates > _______________________________________________ _______________________________________________ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. Follow KRvW Associates on Twitter at: http://twitter.com/KRvW_Associates _______________________________________________
