John Kulig gives some excellent advice on exam construction.
I take exception to two of his suggestions.
First, although obviously wrong answers reduce the number of viable options
and make a guessing strategy more likely to succeed, I like to include one
or two questions with one humorous and obviously-wrong alternative because
these questions help break the tension for students. I also include some
3-alternative questions even though most questions have 4 alternatives
because sometimes I can't come up with a 4th alternative that isn't totally
bogus (and my ability to produce humorous alternatives is limited). Some
researchers suggest that tests comprised of 3-alternative questions achieve
the same level of validity as those comprised of 4- and 5-alternative
questions. (Trevisan et al. 1991)
I usually comply with gambler's fallacy constraints when randomizing
alternatives for an exam. I try to avoid too many stings of the same
response. When I have a large class and I let the test making program
order the questions to create multiple forms, the random process frequently
produces long strings (4 - 6 items) with the same answer. This makes
students crazy. They begin talking themselves out of correct answers just
to break the seemingly non-random pattern. This seems to get in the way of
thinking clearly about the alternatives, so I try to avoid playing this
particular mind game with them. When I order iems myself I now loosen up
my constraints on stings of identical answers & will generate tests with 3
or 4 same-letter answers. A truly random test-making program will produce
tests with unequal numbers of a, b, c, and d answers but I also constrain
this process to make sure that each answer occurs equally often across the
test.
Claudia Stanny
Trevisan, M. S., Sax, G., & Michael, W. B. (1991). The effects of the
number of options per item and student ability on test validity and
reliability. Educational & Psychological Measurement, 51, 829-837.
________________________________________________________
Claudia J. Stanny, Ph.D. e-mail: [EMAIL PROTECTED]
Department of Psychology Phone: (850) 474 - 3163
University of West Florida FAX: (850) 857 - 6060
Pensacola, FL 32514 - 5751
Web: http://www.uwf.edu/~psych/stanny.html
