Gene Gallagher wrote:
>
> Those familiar with "regression to the mean" know what's coming next.
> The poor schools, many in urban centers like Boston, met their
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> improvement "targets," while most of the state's top school districts
> failed to meet their improvement targets.
Wait one... Regression to the mean occurs because of the _random_
component in the first measurement. Being in an urban center is not part
of the random component - those schools' grades didn't improve because
some of them woke up one day and found that their school had moved to a
wealthier district.
If the effect of nonrandom components such as this is large enough
(as I can well believe) to justify the generalization highlighted above,
and if there was a strong pattern of poor-performing schools meeting
their
targets and better-performing schools not doing so, we are looking at
something else - what, I'll suggest later
> The Globe article describes how superintendents of high performing
> school districts were outraged with their failing grades, while the
> superintendent of the Boston school district was all too pleased with
> the evaluation that many of his low-performing schools had improved:
>
> [Brookline High School, for example, with 18 National Merit Scholarship
> finalists and the highest SAT scores in years, missed its test-score
> target - a characterization blasted by Brookline Schools Superintendent
> James F. Walsh, who dismissed the report.
There *is* a problem here,but it's not (entirely) regression
to the mean. If I recall correctly, Brookline High School is
internationally
known as an excellent school, on the basis of decades of excellent
teaching.
If it couldn't meet its target, it's not because its presence among the
top
schools was a fluke in the first measurement - it's probably because the
targets for the top schools were unrealistic.
Was there any justification for the assumption voiced by the Boston
superintendant that the top-performing schools were in fact not
performing at
their capacity and would be "smug" if they assumed that their present
per-
formance was acceptable? The targets described seem to imply that no
school
in the entire state - not one - was performing satisfactorily, even the
top
ones. Perhaps this was felt to be true, or perhaps it was politically
more
acceptable to say "you all need to pull your socks up" than to say "the
following schools need to pull their socks up; the rest of you, steady
as she goes."
As a reductio ad absurdum, if this policy were followed repeatedly,
it would be mathematically impossible for any school to meet its target
every year. That - and not regression to the mean - is the problem
here, I
think.
-Robert Dawson
=================================================================
Instructions for joining and leaving this list and remarks about
the problem of INAPPROPRIATE MESSAGES are available at
http://jse.stat.ncsu.edu/
=================================================================
