Hi,

I as pleasantly surprised to see this topic of discussion flare up
again, because it is one that is much on our mind.

<rant>
In an ideal world, exams are 100% perfect, i.e., everyone who has the
knowledge and skills that the exam tests passes, and the rest fails. In
such a world, a certificate can be used to predict someones abilities:
the holder of the certificate is guaranteed to have the abilities, and
the rest might or might not, depending on whether they ever took the
exam (and failed).

Unfortunately, such the world is not :-).

There are limits to the knowledge and skills you can test. The source of
these limits are manifold, but they include time, money, but also limits
as a result of they way you test. There is only so much you can test in
a multiple choice exam!

So, then the exam becomes a stochastic process: some people who should
fail, pass (a false positive), and some who should pass, fail (a false
negative). As mathematicians can explain you, when you take measures to
decrease the number of false positives, you will almost automatically
increase the number of false negatives! Thus, in order to keep the
number of disappointed people on this planet down (the false negatives),
we have to take a number of false positives (i.e. nitwits with a
certificate) for granted.

But, see social and economic factors kick in!

As more weight is attached to the certificate, the social and economic
desirability of the certificate increases: more people will try and
acquire the certificate. Mathematics tells us that this gives us a
greater number of false positives (as an absolute, not as a percentage),
stated differently: even more nitwits with a certificate. Selective
memory then kicks in: everyone has met a nitwit with a certificate and
relates of this experience whenever suitable (and often when not :-). It
is easier to forget all the able people who happened to have a
certificate. The mere fact that false positives exist somewhat
diminishes the value of a certificate for predicting someone's skills.
However, I think social factors account for a larger diminuition than is
mathematically justified.

However, I think another factor comes into play.

When the economic/social desirability of a certificate becomes great
enough, people will start wanting the certificate not for what it stands
for, but for what it gives them. Their goal becomes not knowing the
things and attaining the skills that are "behind" the certificate, but
knowing the things and attaining the skills that allows them to pass the
exam. Put simpler: they will explicitly start training to pass the exam!
I know that in the MCSE domain there are exam trainings that focus on
the skills needed to pass the exams: exam structure, question structure,
sample exams, exam strategies and so forth.

I have encountered people in an MCSE training whose sole motivation for
following the training (and getting the certificate) was that it would
allow their employer to boost their hourly rate with 20 guilders (+/-
US$ 9,=)! I taught a couple of so called "master classes" that would
lead people to the MCSE status in three months. Some of these folks were
pretty bright and knew how to pass exams: they could study towards that
goal, maintain most of the necessary knowledge in short-termish memory,
pass the exam and then do a memory flush so that there was enough space
left for the next topic of discussion. And, believe me, these are not
isolated incidents!!!!!!

Well, such is life, and it gets sucher every day :-)

We would not want a certification program that only Linus himself can
pass? Would we? 

Life is not perfect, let's just do our best.

</rant>

Jos Visser

 


-- 
Broaden your education by reading: http://www.musa.org.
begin:vcard 
n:Visser;Jos
tel;fax:+31-20-4950 223
tel;home:+31-36-522 0400
tel;work:+31-20-4950 222
x-mozilla-html:FALSE
url:http://www.osp.nl/~josv
org:Open Solution Providers
adr:;;Dalsteindreef 16;Diemen;;NL-1112 XC;Kingdom of the Netherlands
version:2.1
email;internet:[EMAIL PROTECTED]
x-mozilla-cpt:;0
fn:Jos Visser
end:vcard

Reply via email to