In article <[EMAIL PROTECTED]>,
[EMAIL PROTECTED] (Robert J. MacG. Dawson) wrote:
> >The school results are presented in a very odd fashion, making it
> >difficult to assess the patterns.
> >http://www.doe.mass.edu/ata/ratings00/SPRPDistribTables.html
>
> They are that. Let's try.
>
> These data don't look at all like the newspaper story. Here they
are,
> with outcomes given as proportions of each group.
>
> Failed App Met Exceeded N
>
> (Very high 0% 50% 33% 17% 6)
> High 25% 13% 26% 36% 140
> Moderate 43% 17% 23% 17% 471
> LOw 60% 14% 18% 8% 545
> VeryLow 76% 8% 14% 4% 287
> Critical 91% 4% 3% 1% 91
>
> Overall - regression to the mean would cause a NW-SE ridge in
this
> table - and the newspaper story suggested this. What we see is a NE-SW
> ridge. Whatever is causing that ridge is much stronger than regression
> to the mean.
>
> It might be just the demands for more improvement from the "low"
> schools.
>
> To check this, put all the groups onto one quantile plot (I have
> omitted the tiny "very high" group; others are labelled as Critical,
> Very low, Low, Medium, or High by initial):
>
> Proportion of schools improved by fewer than X pts
>
> 0 1 2 3 4 5 6 7 8
>
> 1 v c
> c
> .9 c l
> vm
> .8
> vl
> .7
> h
> .6 lm
>
> .5
> m
> .4 h
>
> .3
> h
> .2
>
> .1
>
> 0
> 0 1 2 3 4 5 6 7 8
>
> Bingo - one common curve, as near as we can tell. We have
different
> groups in different parts of the plot because of the funny way the
data
> were presented, but one curve seems to fit nicely.
>
> This suggests that - far from better schools being penalized for
> regression to the mean, or poorer schools being rewarded for it - the
> ability of schools to improve on a once-off basis was roughly constant
> across the spectrum, and historically poorer-performing schools are
> being penalized by unreasonably-high goals.
>
> What would be reasonable goals? On a once-off basis, it suggests
that
> about 50% of schools can improve by 2 points, about 75% can at least
> hold their own, and (assuming approximate symmetry in the ogive) very
> few schools "in control" would drop by more than about 2 points. A
> possible system, then, would be to give - across the board - a major
pat
> on the back for an improvement of more than 4 points, a minor pat on
the
> back for an improvement of more than 2 points, and an investigation
for
> a drop of more than 2 points. Also, a complementary system based on
raw
> performance.
>
> -Robert Dawson
>
What a wonderful analysis with poorly tabulated data. I'll have to
spend some time seeing how you could dig this point improvement patterns
out of the tables published by DOE. I agree with you that imposing a
6-point improvement scale on the poorest performing schools is an
unrelatistic goal.
--
Eugene D. Gallagher
ECOS, UMASS/Boston
Sent via Deja.com
http://www.deja.com/
=================================================================
Instructions for joining and leaving this list and remarks about
the problem of INAPPROPRIATE MESSAGES are available at
http://jse.stat.ncsu.edu/
=================================================================
