Re: [SC-L] seeking hard numbers of bug fixes...
On Mon, Feb 22, 2010 at 10:45:02AM -0500, Jeremy Epstein wrote: > Take a look at Mary Ann Davidson's keynote at ACSAC in Dec 2009. > http://www.acsac.org/2009/program/keynotes/davidson.pdf This provides a pretty good examination of the costs of patching commercial software. Has anyone done a similar analysis for web applications? I'd expect the costs to be dramatically lower, given thant you're typically producing a single patch for a handful of homogenous systems. -Jon signature.asc Description: Digital signature ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. ___
Re: [SC-L] seeking hard numbers of bug fixes...
Ah, excellent - very helpful! It appears that Laurie Williams at NCSU has inherited John Musa's Software Reliability Engineering legacy, and is still active in the field, and has a number of relevant security articles/papers listed under Publications. http://collaboration.csc.ncsu.edu/laurie/ On 2/22/10 11:22 AM, Wall, Kevin wrote: > Benjamin Tomhave wrote: >> ... we're looking for hard research or >> numbers that covers the cost to catch bugs in code pre-launch and >> post-launch. The notion being that the organization saves itself money >> if it does a reasonable amount of QA (and security testing) >> up front vs trying to chase things down after they've been identified >> (and possibly exploited). > > Ben, > > Not sure if this is what you are looking for or not, but back in the > mid- to late-1980s or so, John Musa, a DMTS at Bell Labs, wrote up a > couple of papers that showed this data, although this was in the more > general context of software quality assurance and not specific to > security testing. > > I'm pretty sure that Musa published something in either one of the ACM > or IEEE CS journals and included some hard data, collected from a bunch > of (then AT&T) Bell Labs projects. IIRC, the main finding was something > like the cost was ~100 times more to catch and correct a bug during > the normal design / coding phase than it was to catch / correct it > after post-deployment. > > Can't help you much more than that. I'm surprised I remembered that much! :) > > -kevin > --- > Kevin W. Wall Qwest Information Technology, Inc. > kevin.w...@qwest.comPhone: 614.215.4788 > "It is practically impossible to teach good programming to students > that have had a prior exposure to BASIC: as potential programmers > they are mentally mutilated beyond hope of regeneration" > - Edsger Dijkstra, How do we tell truths that matter? > http://www.cs.utexas.edu/~EWD/transcriptions/EWD04xx/EWD498.html > > > > This communication is the property of Qwest and may contain confidential or > privileged information. Unauthorized use of this communication is strictly > prohibited and may be unlawful. If you have received this communication > in error, please immediately notify the sender by reply e-mail and destroy > all copies of the communication and any attachments. > > -- Benjamin Tomhave, MS, CISSP tomh...@secureconsulting.net Blog: http://www.secureconsulting.net/ Twitter: http://twitter.com/falconsview LI: http://www.linkedin.com/in/btomhave [ Random Quote: ] "Happiness makes up in height for what it lacks in length." Robert Frost ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. ___
Re: [SC-L] seeking hard numbers of bug fixes...
Benjamin Tomhave wrote: > ... we're looking for hard research or > numbers that covers the cost to catch bugs in code pre-launch and > post-launch. The notion being that the organization saves itself money > if it does a reasonable amount of QA (and security testing) > up front vs trying to chase things down after they've been identified > (and possibly exploited). Ben, Not sure if this is what you are looking for or not, but back in the mid- to late-1980s or so, John Musa, a DMTS at Bell Labs, wrote up a couple of papers that showed this data, although this was in the more general context of software quality assurance and not specific to security testing. I'm pretty sure that Musa published something in either one of the ACM or IEEE CS journals and included some hard data, collected from a bunch of (then AT&T) Bell Labs projects. IIRC, the main finding was something like the cost was ~100 times more to catch and correct a bug during the normal design / coding phase than it was to catch / correct it after post-deployment. Can't help you much more than that. I'm surprised I remembered that much! :) -kevin --- Kevin W. Wall Qwest Information Technology, Inc. kevin.w...@qwest.comPhone: 614.215.4788 "It is practically impossible to teach good programming to students that have had a prior exposure to BASIC: as potential programmers they are mentally mutilated beyond hope of regeneration" - Edsger Dijkstra, How do we tell truths that matter? http://www.cs.utexas.edu/~EWD/transcriptions/EWD04xx/EWD498.html This communication is the property of Qwest and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments. ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. ___
Re: [SC-L] seeking hard numbers of bug fixes...
Take a look at Mary Ann Davidson's keynote at ACSAC in Dec 2009. http://www.acsac.org/2009/program/keynotes/davidson.pdf On Mon, Feb 22, 2010 at 9:17 AM, Benjamin Tomhave wrote: > Howdy, > > This request is a bit time critical as it's supporting a colleague's > upsell up the food chain tomorrow... we're looking for hard research or > numbers that covers the cost to catch bugs in code pre-launch and > post-launch. The notion being that the organization saves itself money > if it does a reasonable amount of QA (and security testing) up front vs > trying to chase things down after they've been identified (and possibly > exploited). > > Any help? > > Thank you, > > -ben > > -- > Benjamin Tomhave, MS, CISSP > tomh...@secureconsulting.net > Blog: http://www.secureconsulting.net/ > Twitter: http://twitter.com/falconsview > LI: http://www.linkedin.com/in/btomhave > > [ Random Quote: ] > "Imagination is everything. It is the preview of life's coming attractions." > Albert Einstein > ___ > Secure Coding mailing list (SC-L) SC-L@securecoding.org > List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l > List charter available at - http://www.securecoding.org/list/charter.php > SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) > as a free, non-commercial service to the software security community. > ___ > ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. ___
