Re: [SC-L] "Checklist Manifesto" applicability to software security
hi sc-l, I am pretty sure that Brian Chess used to have this in his standard talk some many years ago. Then again I am getting old. Great analogy. Note that checklists DO NOT take the place of the intensive care staff! gem On 1/7/10 10:11 AM, "Jeremy Epstein" wrote: Greetings, I was listening yesterday to an interview [1] on NPR with Dr. Atul Gawande, author of "Checklist Manifesto" [2]. He describes the problem that medical procedures (e.g., surgery) tend to have lots of mistakes, mostly caused because of leaving out important steps. He claims that 2/3 of medical - or maybe surgical - errors can be avoided by use of checklists. Checklists aren't very popular among doctors, because they don't like to see themselves as factory workers following a procedure, because the human body is extremely complex, and because every patient is unique. So as I was listening, I was thinking that many of the same things could be said about software developers and problems with software security - every piece of software is unique, any non-trivial piece of software is amazingly complex, developers tend to consider themselves as artists creating unique works, etc. Has anyone looked into the parallelisms before? If so, I'd be interested in chatting (probably offlist) about your thoughts. --Jeremy [1] Listen to the interview at http://wamu.org/programs/dr/10/01/06.php#29280 [2] "The Checklist Manifesto: How to Get Things Right", Atul Gawande, Metropolitan Books. ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. ___ ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. ___
Re: [SC-L] "Checklist Manifesto" applicability to software security
On Thu, Jan 7, 2010 at 7:11 AM, Jeremy Epstein wrote: > Greetings, > > So as I was listening, I was thinking that many of the same things > could be said about software developers and problems with software > security - every piece of software is unique, any non-trivial piece of > software is amazingly complex, developers tend to consider themselves > as artists creating unique works, etc. > > Has anyone looked into the parallelisms before? If so, I'd be > interested in chatting (probably offlist) about your thoughts. I've had exceptionally good luck/results from checklists during the development process, though nothing I could scientifically quantify. That said, I wonder whether any of the academics on the list would be willing to actually do a study. Do some actual trials on defect rates in things like student assignments when they have some students go through a checklist to examine their code, and others not. Might be interesting to see exactly what types of checklist items really result in a reduction in bugs... -- Andy Steingruebl stein...@gmail.com ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. ___
Re: [SC-L] "Checklist Manifesto" applicability to software security
I think there's lots of applicability. People - especially techies - cut corners. The pressure is usually to get things done in a certain amount of time, and then add on that people like to generally expend as little energy as possible, and viola! you see the problem. Of course, the flip side is that checklists in an area like IT can be detrimental, too. PCI is a great example, where it never made a claim of being comprehensive, yet is treated as such (and codified in State laws for crying out loud), and then orgs still get hacked, leaving them to wonder why the checklist didn't protect them. Perhaps the key, then, is knowing that you need experience+procedures. Procedures allow you to not screw up the mundane and routine, while experience allows you to dynamically respond to issues that don't fit the precise steps of the procedure. Part and parcel to this, then, is needing to empower experienced professionals to be flexible and dynamic in the vast of challenges rather than requiring them to rigidly adhere to procedure in all instances. Within appsec, QA and related security testing is probably a great example. If all QA could be strictly proceduralized, then you could just automate it all. However, testing doesn't always go as expected, requiring a functioning brain to (hopefully) respond and adapt accordingly. You probably need procedures for properly catching those exceptions, but nonetheless, those procedures automatically create a capacity for dynamic response. Sorry, a bit rambly... -ben Jeremy Epstein wrote: > Greetings, > > I was listening yesterday to an interview [1] on NPR with Dr. Atul > Gawande, author of "Checklist Manifesto" [2]. He describes the > problem that medical procedures (e.g., surgery) tend to have lots of > mistakes, mostly caused because of leaving out important steps. He > claims that 2/3 of medical - or maybe surgical - errors can be avoided > by use of checklists. Checklists aren't very popular among doctors, > because they don't like to see themselves as factory workers following > a procedure, because the human body is extremely complex, and because > every patient is unique. > > So as I was listening, I was thinking that many of the same things > could be said about software developers and problems with software > security - every piece of software is unique, any non-trivial piece of > software is amazingly complex, developers tend to consider themselves > as artists creating unique works, etc. > > Has anyone looked into the parallelisms before? If so, I'd be > interested in chatting (probably offlist) about your thoughts. > > --Jeremy > > [1] Listen to the interview at http://wamu.org/programs/dr/10/01/06.php#29280 > [2] "The Checklist Manifesto: How to Get Things Right", Atul Gawande, > Metropolitan Books. > ___ > Secure Coding mailing list (SC-L) SC-L@securecoding.org > List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l > List charter available at - http://www.securecoding.org/list/charter.php > SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) > as a free, non-commercial service to the software security community. > ___ > > -- Benjamin Tomhave, MS, CISSP tomh...@secureconsulting.net Blog: http://www.secureconsulting.net/ Twitter: http://twitter.com/falconsview LI: http://www.linkedin.com/in/btomhave [ Random Quote: ] Pareto Principle (a.k.a. The 80-20 Rule): "For many phenomena, 80% of consequences stem from 20% of the causes." http://globalnerdy.com/2007/07/18/laws-of-software-development/ ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. ___
Re: [SC-L] "Checklist Manifesto" applicability to software security
I think it's a great analogy. If you'd like to read more without ordering the book, here's an article Gawande wrote for the New Yorker in 2007: http://www.newyorker.com/reporting/2007/12/10/071210fa_fact_gawande Brian On 1/7/10 7:11 AM, "Jeremy Epstein" wrote: > Greetings, > > I was listening yesterday to an interview [1] on NPR with Dr. Atul > Gawande, author of "Checklist Manifesto" [2]. He describes the > problem that medical procedures (e.g., surgery) tend to have lots of > mistakes, mostly caused because of leaving out important steps. He > claims that 2/3 of medical - or maybe surgical - errors can be avoided > by use of checklists. Checklists aren't very popular among doctors, > because they don't like to see themselves as factory workers following > a procedure, because the human body is extremely complex, and because > every patient is unique. > > So as I was listening, I was thinking that many of the same things > could be said about software developers and problems with software > security - every piece of software is unique, any non-trivial piece of > software is amazingly complex, developers tend to consider themselves > as artists creating unique works, etc. > > Has anyone looked into the parallelisms before? If so, I'd be > interested in chatting (probably offlist) about your thoughts. > > --Jeremy > > [1] Listen to the interview at http://wamu.org/programs/dr/10/01/06.php#29280 > [2] "The Checklist Manifesto: How to Get Things Right", Atul Gawande, > Metropolitan Books. > ___ > Secure Coding mailing list (SC-L) SC-L@securecoding.org > List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l > List charter available at - http://www.securecoding.org/list/charter.php > SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) > as a free, non-commercial service to the software security community. > ___ ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. ___
[SC-L] "Checklist Manifesto" applicability to software security
Greetings, I was listening yesterday to an interview [1] on NPR with Dr. Atul Gawande, author of "Checklist Manifesto" [2]. He describes the problem that medical procedures (e.g., surgery) tend to have lots of mistakes, mostly caused because of leaving out important steps. He claims that 2/3 of medical - or maybe surgical - errors can be avoided by use of checklists. Checklists aren't very popular among doctors, because they don't like to see themselves as factory workers following a procedure, because the human body is extremely complex, and because every patient is unique. So as I was listening, I was thinking that many of the same things could be said about software developers and problems with software security - every piece of software is unique, any non-trivial piece of software is amazingly complex, developers tend to consider themselves as artists creating unique works, etc. Has anyone looked into the parallelisms before? If so, I'd be interested in chatting (probably offlist) about your thoughts. --Jeremy [1] Listen to the interview at http://wamu.org/programs/dr/10/01/06.php#29280 [2] "The Checklist Manifesto: How to Get Things Right", Atul Gawande, Metropolitan Books. ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. ___
