Asking about security in terms of an RFP is a big joke and reminds me of tactics I used in sixth grade when I used to figure out creative ways of answering a question by turning the question into an answer. One has to acknowledge that RFPs are not authoratative and are usually completed by sales teams who don't have adequate detail.
So, instead of focusing on comprehensive documentation, you need to be agile and think more about working software. I believe that penetration tests are sadly too late in the process in order to be meaningful and only have value in scenarios where you can mandate that the presses be stopped and that they fix immediately before you give them a single penny. Avoid the contract stuff as well as you can't really put meaningful things into agreements. You have to acknowledge that if I were successful in putting say a clause into our next EMC agreement requiring all document management software to support XACML and be OWASP Top Ten free, do you think that a developer on the other end would even have a chance to read it and address going forward? Way too many degrees of separation between those who write contracts and those who implement software. Again, I think you need to measure developers and their participation in groups such as OWASP since it is observable and measurable without requiring a lot of effort. It is not a guarantee but can be a predictor... -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Herman Stevens Sent: Monday, December 01, 2008 1:55 PM To: Marcin Wielgoszewski Cc: SC-L@securecoding.org Subject: Re: [SC-L] FW: How Can You Tell It Is Written Securely? Hello Marcin, I agree with your statement that many companies have some requirements in their SLA's with outsourced development firms. However, if these are really F-100 businesses they usually have all non-core processes out-sourced (because a Big4 company told them that would reduce costs), the relationship management with the outsourced companies is also out-sourced (probably to the same Big4). This means after a few years all knowledge has left the company and if a Request For Proposal needs to be written (e.g. for a new application supporting their core business functions) this is outsourced again to the same Big4 since the company itself does not even have the required knowledge to write its own RFPs .. I really doubt that anything that goes in that RFP (and ultimately in the contracts) will have any _real_ security value. Using penetration tests and vulnerability requirements might be part of the acceptance process, but I do not call these tests _security_ requirements. They are acceptance requirements ... The original request asked for how can someone determine if an application is written in a secure manner. My reasoning is that this is the wrong question (the application must _be_ secure and for this there is no direct link with coding practices). And even if one can proof the application is written in a secure manner, this will not be enough to be secure (e.g. about 99% of all security relevant features are nowadays in the configuration, the customer will never issue a change request for a new java library of javascript library yet in many of my penetration tests I 'break' the application because of old libs, ...). I do not think that penetration tests and vulnerability assessments are a 'proof' that an application is written securely. I've seen many applications that were written horrendously but were very secure (in the sense that they abided to all security-relevant business requirements) and I have seen many applications written using the 'best practices' in coding and developed with very mature processes that could be hacked in minutes. So, are there any studies that proof that a company that performs some tests (e.g. pen-tests) or include security requirements in the contracts ultimately is better off than a company that does not do what we consider 'best practices'? And if we don't have that proof, shouldn't we be very prudent in what we advise to our customers? Please note that my company sells security related software and performs vulnerability assessments, so I'm not saying that these are useless (:)), but maybe there are better methods than penetrate & patch or enforcing very heavy processes on innocent development teams... So, this is question to this list: Are we on the right track? Is application security really improving? Do we measure the correct things and in the correct way? My point of view is that only certain vulnerabilities are less common than in the early days just because of more mature frameworks, but not due to better processes or after the fact testing. Does this mean all efforts were vain? Or did the threat landscape change? And yes, there are many vendor driven statistics floating around but they really cannot be considered unbiased ... Lots of questions, maybe not all relevant for the Secure Coding list, but Secure Coding should have an final objective. Or not? Herman [EMAIL PROTECTED] ************************************************************ This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies. ************************************************************ _______________________________________________ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. _______________________________________________