Eric Bohlman wrote:
> Robert J. MacG. Dawson <[EMAIL PROTECTED]> wrote:
> > If indeed the scores are being reduced by hiding the easy questions
> > among the harder ones, then I would say yes, this is a defect of the
> > current system, and should be changed. It may be that the questions
> > themselves ought to be more difficult; but the difficulty ought to be
> > intrinsic to the questions, not an artifact of the test format. What is
> > at issue here is essentially signal-to-noise ratio.
>
> In fact, format-related "difficulty" on a test gives an unfair advantage
> to students who have taken test-prep courses, since most of the skills
> they teach relate to pacing, figuring out formats, and the like rather
> than to actual academic content. Since test-taking skills are generally
> *not* taught in schools themselves,
Not so fast, gang! Since schools (and sometimes teachers) are rated on the
percent of students who pass a standardized test, it has become common in many
places for teachers to spend a good deal of time teaching the kids how to take
the tests. In the elementary grades this can amount to significant
percentages of 'education' time. In some places kids had to forego recess to
get in the exam prep time. That's why they call them high stakes tests - the
outcome is critical to teachers, principals, and superintendents' livelihood.
As we (collective we) pointed out, a disparity in test taking preparation
distorts the scores achieved, clouding (through reduced signal/noise ratio)
the interpretability of the results. Did someone in authority, who ordered
these high stakes tests, _not_ expect this distortion to occur? It may be an
unintended consequence, but I can hardly believe it was unexpected.
The distortion of the educational curriculum content is another consequence.
So long as the questions have the same relationship to technical content as
the famous MCAS #39 has to what we call statistical thinking (and doing), we
can expect more and continuing distortions of education content. What teacher
would struggle to understand and communicate the real thing, when the test
won't come close to it? If the students do not do well on the test, said
teacher can kiss their job goodby. Time to get with the program.
that's for public schools. In most states, the high stakes tests are not
administered, or reported, by private schools. Read that for what you will.
> evaluating schools on criteria that
> are strongly influenced by their students' test-taking skills amounts to
> evaluating them on criteria beyond their control, in this case the number
> of parents who both care enough and (*big* and!) can afford to send their
> kids to test-prep courses.
>
Jay
--
Jay Warner
Principal Scientist
Warner Consulting, Inc.
4444 North Green Bay Road
Racine, WI 53404-1216
USA
Ph: (262) 634-9100
FAX: (262) 681-1133
email: [EMAIL PROTECTED]
web: http://www.a2q.com
The A2Q Method (tm) -- What do you want to improve today?
=================================================================
Instructions for joining and leaving this list and remarks about
the problem of INAPPROPRIATE MESSAGES are available at
http://jse.stat.ncsu.edu/
=================================================================
