> I agree that multiple choice alone is inadequate to test the true > breadth and depth of someone's security knowledge. Having contributed > a few questions to the SANS pool, I take issue with Gary's article > when it implies that you can pass the GSSP test while clueless. > > There is indeed a body of knowledge that is being tested. SANS has > been soliciting comments on the document.
Having taught this type of material before at the university and vocational levels, I think there are three main aspects which are important to someone's capability to code "securely": 1 - Knowledge of pitfalls, countermeasures, and good practices; 2 - The right mindset; and 3 - Experience carrying it out (there are also the surrounding business issues, like project management and planning, risk assessment and how well-vetted the software must be given the cost/risk scenario, but I'll just stick to the coder for now). I have not reviewed the GSSP (practice exams, etc.), but I am guessing that it goes after the "low hanging fruit" of covering (1) above, which is testable most easily with an exam. It's much better than nothing, and the knowledge is very important, but this test does not necessarily mean that a particular coder will be a better "secure coder". There's a lot more to this than just a body of knowledge. For example, you could give any auto mechanic a test that they could pass if they know what the risks are in leaving a bolt loose, or a fuel system clamp unsecured, or not replacing an O-ring when a connection is open (or, if they can figure out those risks during the exam, esp. a multiple-choice one). But that does not mean that the mechanic will actually follow through with those things, or that, in practice, the mechanic will actually be more prone to even notice... So, although I think the GSSP is an important first step, I tend to agree with Gary. In my university-level teaching of software security, I would never even begin to consider evaluating my students merely via multiple choice exams. Not with this subject matter. Greg. _______________________________________________ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. _______________________________________________
