Lucas wrote:
either you have the knowledge or not and if you have it you don't need to choose
the right answer because you know it.
True, but multiple choice also has its advantages. It's a lot cheaper,
To be specific, the answers to multiple choice answers can be highly
automated, to the point where an exam taken at a VUE or Prometric centre
can offer a pass/fail score in real time upon completion. Diagrams,
essay questions, etc. require a warm body to grade -- how much will that
add to the cost of an exam? Can you really count on a volunteer
infrastructure?
Also, it's important to consider how well a scoring process will scale
as numbers increase. LPI's just purchased a scanner so that scoring
speed (and capacity) can be increased from six per hour to six per minute.
There's a reason the "hands-on"Red Hat RHCE exam costs $700 -- and
they're not making much money on that. RH's real money is in education,
certification is just a tool (to them, like to many software vendors who
do certs) that helps drive training sales.
A reasonable compromise is the "fill in the blank" item, which can still
be automated while not spoonfeeding possible answers to candidates. Even
if a FITB question has multiple possible answers (and most do), that can
be accommodated in automated scoring. (Aren't regular expressions
wonderful?)
Besides, you can make multiple choice questions that
need thorough understanding of the subject matter.
Very true. It's a big challenge, but a well-crafted exam containing
multiple-choice, multiple-choice-multi-answer and FITB items can perform
very well. "Deceptors" (the psychometric term for incorrect options in
a MC item) should be plausible yet not ambiguously wrong.
There is *no* science that proves hands-on exams actually do a better
job of separating skilled from unskilled compared to hands-on exams.
OTOH, hands-on exams have problems with reliability, sufficient that
they can't be psychometrically validated.
- Evan
_______________________________________________
BSDCert mailing list
[email protected]
http://lists.nycbug.org/mailman/listinfo/bsdcert
