[SC-L] ZDNET: LAMP lights the way in open-source security
Interesting article out on ZDNet today: http://www.zdnetasia.com/news/security/0,39044215,39315781,00.htm The article refers to the US government sponsored study being done by Stanford University, Symantec, and Coverity. It says, The so-called LAMP stack of open-source software has a lower bug density--the number of bugs per thousand lines of code--than a baseline of 32 open-source projects analyzed, Coverity, a maker of code analysis tools, announced Monday. This surprised me quite a bit, especially given LAMP's popular reliance on scripting languages PHP, Perl, and/or Python. Still, the article doesn't discuss any of the root causes of the claimed security strengths in LAMP-based code. Perhaps it's because the scripting languages tend to make things less complex for the coders (as opposed to more complex higher level languages like Java and C#/.NET)? Opinions? Cheers, Ken -- Kenneth R. van Wyk KRvW Associates, LLC http://www.KRvW.com ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php
RE: [SC-L] ZDNET: LAMP lights the way in open-source security
The Coverity product (Coverity Prevent) is a static source code analysis tool for C and C++, see http://www.coverity.com/library/pdf/coverity_prevent.pdf. It isn't actually scanning (or if it is, it isn't analyzing) any of the scripting code, as far I as can tell. Michael -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Kenneth R. van Wyk Sent: Tuesday, March 07, 2006 10:56 AM To: Secure Coding Mailing List Subject: [SC-L] ZDNET: LAMP lights the way in open-source security Interesting article out on ZDNet today: http://www.zdnetasia.com/news/security/0,39044215,39315781,00.htm The article refers to the US government sponsored study being done by Stanford University, Symantec, and Coverity. It says, The so-called LAMP stack of open-source software has a lower bug density--the number of bugs per thousand lines of code--than a baseline of 32 open-source projects analyzed, Coverity, a maker of code analysis tools, announced Monday. This surprised me quite a bit, especially given LAMP's popular reliance on scripting languages PHP, Perl, and/or Python. Still, the article doesn't discuss any of the root causes of the claimed security strengths in LAMP-based code. Perhaps it's because the scripting languages tend to make things less complex for the coders (as opposed to more complex higher level languages like Java and C#/.NET)? Opinions? Cheers, Ken -- Kenneth R. van Wyk KRvW Associates, LLC http://www.KRvW.com ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php
RE: [SC-L] ZDNET: LAMP lights the way in open-source security
Yeah, statistics can allow you to say and prove just about anything. OK, showing my ignorance here, since I haven't checked out any of the LAMP source trees and reviewed the code: how much of the code making up those modules is written in scripting languages vs. how much of it is written in C, C++ (and how much, if any, is written in any other compiled languages)? If the LAMP source code itself is primarily C/C++, then arguably, the results are somewhat interesting, though I think they would be much more interesting if this DISA project was set up to test the open source code with a number of commercial scanners instead of just the Coverity scanner, then we could at least compare the merits of various scanning techniques and implementations. In this case, the distinction to me is that they have tested the LAMP platform code, not the code that people write on top of it for their applications, and are making some statements about the software security of the LAMP platform compared to the rest of the open source code they scanned. If on the other hand, a significant portion of the LAMP code base itself is made up of scripting language code, then I agree with you, the results aren't terribly useful to anyone other than possibly Coverity and Stanford. Note: significant is open to interpretation, but doesn't have to be large; 10 or 15 per cent would seem significant enough to me. -Original Message- From: Jeremy Epstein [mailto:[EMAIL PROTECTED] Sent: Tuesday, March 07, 2006 12:17 PM To: Gavin, Michael; Kenneth R. van Wyk; Secure Coding Mailing List Subject: RE: [SC-L] ZDNET: LAMP lights the way in open-source security All of which proves that there are lies, damn lies, and statistics (the statistic being the lower bug density, which ignores the most potentially vulnerable parts of the system). -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Gavin, Michael Sent: Tuesday, March 07, 2006 11:49 AM To: Kenneth R. van Wyk; Secure Coding Mailing List Subject: RE: [SC-L] ZDNET: LAMP lights the way in open-source security The Coverity product (Coverity Prevent) is a static source code analysis tool for C and C++, see http://www.coverity.com/library/pdf/coverity_prevent.pdf. It isn't actually scanning (or if it is, it isn't analyzing) any of the scripting code, as far I as can tell. Michael -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Kenneth R. van Wyk Sent: Tuesday, March 07, 2006 10:56 AM To: Secure Coding Mailing List Subject: [SC-L] ZDNET: LAMP lights the way in open-source security Interesting article out on ZDNet today: http://www.zdnetasia.com/news/security/0,39044215,39315781,00.htm The article refers to the US government sponsored study being done by Stanford University, Symantec, and Coverity. It says, The so-called LAMP stack of open-source software has a lower bug density--the number of bugs per thousand lines of code--than a baseline of 32 open-source projects analyzed, Coverity, a maker of code analysis tools, announced Monday. This surprised me quite a bit, especially given LAMP's popular reliance on scripting languages PHP, Perl, and/or Python. Still, the article doesn't discuss any of the root causes of the claimed security strengths in LAMP-based code. Perhaps it's because the scripting languages tend to make things less complex for the coders (as opposed to more complex higher level languages like Java and C#/.NET)? Opinions? Cheers, Ken -- Kenneth R. van Wyk KRvW Associates, LLC http://www.KRvW.com ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php
RE: [SC-L] ZDNET: LAMP lights the way in open-source security
I'm a strong advocate of static analysis, but drawing conclusions about overall security based only on these tools is just silly. Even ignoring the scripting language problem, these tools simply aren't even looking for many of the types of problems that cause the most serious risks. They're great for assisting a code review or indicating potential design flaws, but not a great ruler. At least not yet. --Jeff -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Gavin, Michael Sent: Tuesday, March 07, 2006 12:46 PM To: Jeremy Epstein; Kenneth R. van Wyk; Secure Coding Mailing List Subject: RE: [SC-L] ZDNET: LAMP lights the way in open-source security Yeah, statistics can allow you to say and prove just about anything. OK, showing my ignorance here, since I haven't checked out any of the LAMP source trees and reviewed the code: how much of the code making up those modules is written in scripting languages vs. how much of it is written in C, C++ (and how much, if any, is written in any other compiled languages)? If the LAMP source code itself is primarily C/C++, then arguably, the results are somewhat interesting, though I think they would be much more interesting if this DISA project was set up to test the open source code with a number of commercial scanners instead of just the Coverity scanner, then we could at least compare the merits of various scanning techniques and implementations. In this case, the distinction to me is that they have tested the LAMP platform code, not the code that people write on top of it for their applications, and are making some statements about the software security of the LAMP platform compared to the rest of the open source code they scanned. If on the other hand, a significant portion of the LAMP code base itself is made up of scripting language code, then I agree with you, the results aren't terribly useful to anyone other than possibly Coverity and Stanford. Note: significant is open to interpretation, but doesn't have to be large; 10 or 15 per cent would seem significant enough to me. -Original Message- From: Jeremy Epstein [mailto:[EMAIL PROTECTED] Sent: Tuesday, March 07, 2006 12:17 PM To: Gavin, Michael; Kenneth R. van Wyk; Secure Coding Mailing List Subject: RE: [SC-L] ZDNET: LAMP lights the way in open-source security All of which proves that there are lies, damn lies, and statistics (the statistic being the lower bug density, which ignores the most potentially vulnerable parts of the system). -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Gavin, Michael Sent: Tuesday, March 07, 2006 11:49 AM To: Kenneth R. van Wyk; Secure Coding Mailing List Subject: RE: [SC-L] ZDNET: LAMP lights the way in open-source security The Coverity product (Coverity Prevent) is a static source code analysis tool for C and C++, see http://www.coverity.com/library/pdf/coverity_prevent.pdf. It isn't actually scanning (or if it is, it isn't analyzing) any of the scripting code, as far I as can tell. Michael -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Kenneth R. van Wyk Sent: Tuesday, March 07, 2006 10:56 AM To: Secure Coding Mailing List Subject: [SC-L] ZDNET: LAMP lights the way in open-source security Interesting article out on ZDNet today: http://www.zdnetasia.com/news/security/0,39044215,39315781,00.htm The article refers to the US government sponsored study being done by Stanford University, Symantec, and Coverity. It says, The so-called LAMP stack of open-source software has a lower bug density--the number of bugs per thousand lines of code--than a baseline of 32 open-source projects analyzed, Coverity, a maker of code analysis tools, announced Monday. This surprised me quite a bit, especially given LAMP's popular reliance on scripting languages PHP, Perl, and/or Python. Still, the article doesn't discuss any of the root causes of the claimed security strengths in LAMP-based code. Perhaps it's because the scripting languages tend to make things less complex for the coders (as opposed to more complex higher level languages like Java and C#/.NET)? Opinions? Cheers, Ken -- Kenneth R. van Wyk KRvW Associates, LLC http://www.KRvW.com ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php
RE: [SC-L] ZDNET: LAMP lights the way in open-source security
-Original Message- From: Crispin Cowan [mailto:[EMAIL PROTECTED] Gavin, Michael wrote: Yeah, statistics can allow you to say and prove just about anything. OK, showing my ignorance here, since I haven't checked out any of the LAMP source trees and reviewed the code: how much of the code making up those modules is written in scripting languages vs. how much of it is written in C, C++ (and how much, if any, is written in any other compiled languages)? That doesn't matter; what matters is what fraction of disclosed vulnerabilities is in each segment of the code? If 90% of the vulnerabilities come from the PHP part, then the fact that 90% of the lines of code are in C doesn't help. [Gavin, Michael] Absolutely true! But from the perspective of improving static source code analysis tools, if 90% of the code is in C, which is one of the 2 languages supported by the Coverity product, then we now have one reasonable data point regarding how well that (moderate amount of) C code was written with respect to one vendor's notion/implementation of secure coding in C. Certainly not a huge win for anyone, but a potential starting point for comparing techniques and products. For example, I haven't been following the status of David Wheeler's flawfinder, but even if that hasn't been updated lately, it might be interesting to see which flaws it finds that Coverity found, which Coverity found that flawfinder doesn't, and which flawfinder finds that Coverity didn't. Unfortunately your comment below regarding the proprietary nature of Coverity makes such a comparison less useful for everyone but Coverity... If the LAMP source code itself is primarily C/C++, then arguably, the results are somewhat interesting, though I think they would be much more interesting if this DISA project was set up to test the open source code with a number of commercial scanners instead of just the Coverity scanner, then we could at least compare the merits of various scanning techniques and implementations. The proprietary status of the Coverity scanner is a continuous pain. That's why I tend to ignore it where possible :) Crispin -- Crispin Cowan, Ph.D. http://crispincowan.com/~crispin/ Director of Software Engineering, Novell http://novell.com Olympic Games: The Bi-Annual Festival of Corruption ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php
RE: [SC-L] ZDNET: LAMP lights the way in open-source security
Hmm. Time to no longer use flawfinder, RATS, and ITS4. Throw them out and get a real tool. I cover this in gory detail in chapter 5 of Software Security. There's a pretty nice treatment of the history of these tools and the evolution of technology there. gem www.swsec.com www.cigital.com/~gem -Original Message- From: Gavin, Michael [mailto:[EMAIL PROTECTED] Sent: Tue Mar 07 16:40:00 2006 To: Crispin Cowan Cc: Jeremy Epstein; Secure Coding Mailing List Subject:RE: [SC-L] ZDNET: LAMP lights the way in open-source security -Original Message- From: Crispin Cowan [mailto:[EMAIL PROTECTED] Gavin, Michael wrote: Yeah, statistics can allow you to say and prove just about anything. OK, showing my ignorance here, since I haven't checked out any of the LAMP source trees and reviewed the code: how much of the code making up those modules is written in scripting languages vs. how much of it is written in C, C++ (and how much, if any, is written in any other compiled languages)? That doesn't matter; what matters is what fraction of disclosed vulnerabilities is in each segment of the code? If 90% of the vulnerabilities come from the PHP part, then the fact that 90% of the lines of code are in C doesn't help. [Gavin, Michael] Absolutely true! But from the perspective of improving static source code analysis tools, if 90% of the code is in C, which is one of the 2 languages supported by the Coverity product, then we now have one reasonable data point regarding how well that (moderate amount of) C code was written with respect to one vendor's notion/implementation of secure coding in C. Certainly not a huge win for anyone, but a potential starting point for comparing techniques and products. For example, I haven't been following the status of David Wheeler's flawfinder, but even if that hasn't been updated lately, it might be interesting to see which flaws it finds that Coverity found, which Coverity found that flawfinder doesn't, and which flawfinder finds that Coverity didn't. Unfortunately your comment below regarding the proprietary nature of Coverity makes such a comparison less useful for everyone but Coverity... If the LAMP source code itself is primarily C/C++, then arguably, the results are somewhat interesting, though I think they would be much more interesting if this DISA project was set up to test the open source code with a number of commercial scanners instead of just the Coverity scanner, then we could at least compare the merits of various scanning techniques and implementations. The proprietary status of the Coverity scanner is a continuous pain. That's why I tend to ignore it where possible :) Crispin -- Crispin Cowan, Ph.D. http://crispincowan.com/~crispin/ Director of Software Engineering, Novell http://novell.com Olympic Games: The Bi-Annual Festival of Corruption ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php This electronic message transmission contains information that may be confidential or privileged. The information contained herein is intended solely for the recipient and use by any other party is not authorized. If you are not the intended recipient (or otherwise authorized to receive this message by the intended recipient), any disclosure, copying, distribution or use of the contents of the information is prohibited. If you have received this electronic message transmission in error, please contact the sender by reply email and delete all copies of this message. Cigital, Inc. accepts no responsibility for any loss or damage resulting directly or indirectly from the use of this email or its contents. Thank You. ___ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php
