Before you decide, keep in mind that on some exams, the decision of what approach to use for a problem (higher level than recall) represents a portion of the knowledge we expect to find in a successful student. Putting all the questions from one section of a text into one place means that this decision has been made already - we aren't testing for it. The score achieved by students may well be higher, but the test will not measure the ability of the student to select a method for solution ('path to the solution' is the term I used in physical chemistry/intro thermodynamics classes.) Every instructor has seen this effect, when they give questions for each chapter, get responses, then get blanks from students during the final exam that includes all chapters.
How you structure the test, including the order of the questions, determines in part what you are measuring with the test. Whether MCAS developers included these effects in their deliberations I can't say.
I can argue that ability to select a path to a solution, for any problem/challenge, is a more valuable skill for a HS grad than any degree of recognition and recall. it is also harder to teach, much less test for.
Jay
"Robert J. MacG. Dawson" wrote:
Dennis Roberts wrote:> of course, research eons ago has shown that test performance is optimized
> ... by having items in the order of easy to difficult ... IF there is a
> time limit where some examinees have to push to get finished
>
> now that's a thought ... maybe if the items WERE ordered that way ... some
> of that large % that seem to be failing ... would gain an item or two in
> their score and pass!!! what a simple thing to do to make the students in
> mass. look better! and mass. education!It seems to me that the "advantage" given by putting the problems in
approximate order of difficulty is legitimate - much like the
"advantage" given by having the questions printed in a legible font, by
not interrupting the test several times for corrections and random
information, or by not scheduling a fire drill during the test.If indeed the scores are being reduced by hiding the easy questions
among the harder ones, then I would say yes, this is a defect of the
current system, and should be changed. It may be that the questions
themselves ought to be more difficult; but the difficulty ought to be
intrinsic to the questions, not an artifact of the test format. What is
at issue here is essentially signal-to-noise ratio.-Robert Dawson
=================================================================
Instructions for joining and leaving this list and remarks about
the problem of INAPPROPRIATE MESSAGES are available at
http://jse.stat.ncsu.edu/
=================================================================
--
Jay Warner
Principal Scientist
Warner Consulting, Inc.
4444 North Green Bay Road
Racine, WI 53404-1216
USA
Ph: (262) 634-9100
FAX: (262) 681-1133
email: [EMAIL PROTECTED]
web: http://www.a2q.com
The A2Q Method (tm) -- What do you want to improve today?
=================================================================
Instructions for joining and leaving this list and remarks about
the problem of INAPPROPRIATE MESSAGES are available at
http://jse.stat.ncsu.edu/
=================================================================
