Ruven's answer made an assumption that I think most of us did when reading your question, that you're aiming to evaluate "programmers" as in "those who have the job of programming." Another interpretation of your question (which is why I'm asking for clarification) is that you're interested in evaluating an individual's skill or conceptual background in programming. That's not exactly the same thing.
My work the last few years has been focused on teaching programming skills and computing concepts to non-CS majors. Our patron saint is Alan Perlis, who argued in the early 1960's that *everyone* should learn to program, that it should be part of a general, liberal education. He argued that computer science is the study of process, and everyone is concerned with specifying, understanding, debugging, and implementing process. Our non-CS majors don't have a goal of ever becoming programmers (as in "have the job of programming"), but they might end up doing some programming in their lives (programming in the small, e.g., scripting). To teach "liberal education" programming, our course emphasizes basic concepts of specifying process (e.g., sequencing, conditionals, iteration, functions, data structures such as arrays) in a concrete context of media manipulation (e.g., students create Photoshop-style image filters, reverse and splice sounds, write Web-mining programs, and create animations). (If you're interested, papers on the work are at http://coweb.cc.gatech.edu/mediaComp-plan/Papers)
We're faced with a similar problem as you are -- how do we evaluate what these students know about programming? How does what they learn compare to what CS majors learn in a first course? We can't use Ruven's solution, because we're not teaching design skill or ability to write test plans. Ours are the students who, as you say, will be able to use a sorting algorithm, but probably won't be able to write one.
We're starting a project with Stephen Cooper and Wanda Dann who have the SAME problem. They're teaching programming using the Alice 3-D storytelling environment (http://www.alice.org). Again, their students are learning about those basic notions of specifying process, but not about the ideas that are closer to software engineering skills. We're trying to develop some robust measures that can help us evaluate these students' programming skills.
Now that I've written an overly long request for clarification, could you say some more about who and what you're trying to evaluate?
Thanks! Mark
On Sep 13, 2004, at 4:19 PM, Polly Chase wrote:
Hi, I am curious about how programming ability is normally evaluated. Are there ways to decide how deeply a person understands topics in software development? What are some examples of things a person with a deeper understanding would know? What are examples of things a person with a shallower understanding would fail to grasp? What are typical signs of understanding vs. not understanding? Is it possible to develop and use software without having much awareness of underlying concepts? I realize anyone can use a sorting function without knowing the details of different sorting algorithms, or query a database without knowing all about searching algorithms. But what is the minimal level of awareness needed, and how can the awareness level be estimated? (I realize what I'm asking is very general, and I am only looking for general answers). -pc
---------------------------------------------------------------------- PPIG Discuss List ([EMAIL PROTECTED]) Discuss admin: http://limitlessmail.net/mailman/listinfo/discuss Announce admin: http://limitlessmail.net/mailman/listinfo/announce PPIG Discuss archive: http://www.mail-archive.com/discuss%40ppig.org/
__________ Mark Guzdial : Georgia Tech : College of Computing/GVU Atlanta, GA 30332-0280 Collaborative Software Lab, http://coweb.cc.gatech.edu/csl http://www.cc.gatech.edu/~mark.guzdial/
---------------------------------------------------------------------- PPIG Discuss List ([EMAIL PROTECTED]) Discuss admin: http://limitlessmail.net/mailman/listinfo/discuss Announce admin: http://limitlessmail.net/mailman/listinfo/announce PPIG Discuss archive: http://www.mail-archive.com/discuss%40ppig.org/
