On Thu, 1 Jul 1999, Forrest Tiffany wrote:
> I think this would be VERY important. Oftentimes when writing anything
> there
> is a lack of clarity between what is intended to be expressed, and what
> is
> actually written down. This is true of prose, poetry, programming, and
> yes
> too, test questions. I think the test questions should be regarded like
> any
> software package. They have bugs in them. The bugs just haven't been
> found
> yet, and without sufficient testing on a sufficient number of machines,
> in
> our case people, the bugs won't be found.
You are absolutely correct, Forrest. At their most basic level, test
questions, like lines of C code, are just expressions of thought. Either
can have bugs in them (and they usually do!), so it's important to
carefully analyze them to eliminate the bugs.
However, an important difference between tests and c source files is that
the former are processed by humans and the latter by machines. Thus, it is
theoretically easy to predict how the compiler will react to each line of
code. But it's very hard to predict how each person will react to each
test question. That's why it is especially important that test questions
be reviewed by as many people as possible.
Another point to keep in mind is that while asking people to critique test
questions is important, it's not the end of the game. Like in software
development, what really matters is not the elegance of the source code
but how well it solves the problem it was intended to solve. In test
development, what really matters is how well the questions actually
function in discriminating between those who are qualified and those who
are not. This issue can only be settled with actual empirical data,
responses from a large number of individuals to each question. If a
particular question correlates with competence, then it is a good
question. If it doesn't correlate with competence, no matter how well it
is written and how much the reviewers liked it, it is a bad question.
-Scott
________________________________________________________________________
This message was sent by the linux-cert-program mailing list. To unsubscribe:
echo unsubscribe | mail -s '' [EMAIL PROTECTED]
