Re: MA MCAS statistical fallacy

2001-01-30 Thread dennis roberts
i think that one thing that math class teaches you about "measurement" is that there is error ... well maybe they do, now that i think of it, i am not so sure how clear this notion is taught ... but, let's assume that it is ... math would also reinforce meaning of the use of the english ( in

Re: MA MCAS statistical fallacy

2001-01-30 Thread Rich Ulrich
On 22 Jan 2001 15:58:16 -0800, [EMAIL PROTECTED] (dennis roberts) wrote: > At 03:28 PM 1/22/01 -0500, Rich Ulrich wrote: < snip, details of my alternative examples of statements > > as i said before ... given the stem and the choice C of 1 foot ... i think > any intelligent examinee could argu

Re: MA MCAS statistical fallacy

2001-01-23 Thread Tony T. Warnock
"P.G.Hamer" wrote: > dennis roberts wrote: > > > there just is no good way to argue against the original choice C ... IN THE > > CONTEXT OF THE STEM OF THE QUESTION > > I am reminded of the joke article that contains many `pollitically incorrect' > answers to the exam question "given a baromete

Re: MA MCAS statistical fallacy, rather OT

2001-01-23 Thread Robert J. MacG. Dawson
"P.G.Hamer" wrote: > > dennis roberts wrote: > > > there just is no good way to argue against the original choice C ... IN THE > > CONTEXT OF THE STEM OF THE QUESTION > > I am reminded of the joke article that contains many `politically incorrect' > answers to the exam question "given a barom

Re: MA MCAS statistical fallacy

2001-01-23 Thread P.G.Hamer
dennis roberts wrote: > there just is no good way to argue against the original choice C ... IN THE > CONTEXT OF THE STEM OF THE QUESTION I am reminded of the joke article that contains many `pollitically incorrect' answers to the exam question "given a barometer how do you measure the hight of

Re: MA MCAS statistical fallacy

2001-01-22 Thread dennis roberts
At 03:28 PM 1/22/01 -0500, Rich Ulrich wrote: >"If Matt scored 121 on an IQ test and Damien scored 123 on the same >test, what is the maximum difference in their IQs?" the comparable item to the real one shown before would be: "if matt was measured ACCURATELY (rounded to the nearest single IQ

Re: MA MCAS statistical fallacy

2001-01-22 Thread Rich Ulrich
On 16 Jan 2001 09:01:18 -0800, [EMAIL PROTECTED] (Robert J. MacG. Dawson) wrote: > > > Rich Ulrich wrote: > > > Construing the language as precisely as possible, but being careful to > > take into account the full language of the question and the > > multiple-choice answers, what do you think

Re: MA MCAS statistical fallacy

2001-01-17 Thread dennis roberts
At 03:57 PM 1/17/01 -0500, Rich Ulrich wrote: > - Okay, here is my answer before I repeat the official ones. >The "greatest possible difference" is *at least* one foot. >If this is a dedicated math question, the aspect of roundoff should >give "one foot (minimum)"; and any slightest introduct

Re: MA MCAS statistical fallacy

2001-01-17 Thread Rich Ulrich
On Tue, 16 Jan 2001 14:14:36 GMT, [EMAIL PROTECTED] (J. Williams) concluded: > Maybe, I am missing something, but think the original question and > response items are quite clear and concise. I see nothing > particularly "loose" about it. The essentials of a class interval > used in frequency d

Re: MA MCAS statistical fallacy

2001-01-16 Thread dennis roberts
At 11:33 AM 1/16/01 -0400, you wrote: >> 37. When Matt's and Damien's broad jumps were measured accurately to >> the nearest foot, each measurement was 21 feet. Which statement best >> describes the greatest possible difference in the lengths of MattĀ¹s >> jump and Damien's jump? >> >> A. One jum

Re: AW: MA MCAS statistical fallacy

2001-01-16 Thread Jerry Dallal
Werner Wittmann wrote: > > See the regression artifact primer at Dave's homepage: > http://nw3.nai.net/~dakenny/rrtm.htm I looked at http://nw3.nai.net/~dakenny/primer.htm and found myself puzzled by the Galton squeeze plot (or is it a pair link diagram, or are they one and the same?). It sho

Re: MA MCAS statistical fallacy

2001-01-16 Thread Robert J. MacG. Dawson
Rich Ulrich wrote: > Construing the language as precisely as possible, but being careful to > take into account the full language of the question and the > multiple-choice answers, what do you think the correct answer is? > > Do you think the question is actually OK? Is the wording good enough

Re: MA MCAS statistical fallacy

2001-01-16 Thread J. Williams
On Mon, 15 Jan 2001 19:47:49 -0500, Rich Ulrich <[EMAIL PROTECTED]> wrote: >Concerning the MCAS. There was a discussion last month >in another Usenet group, alt.usage.english, concerning one of >its math questions which was written too loosely. > >Here is the start of that thread. The thread

Re: MA MCAS statistical fallacy

2001-01-15 Thread Rich Ulrich
Concerning the MCAS. There was a discussion last month in another Usenet group, alt.usage.english, concerning one of its math questions which was written too loosely. Here is the start of that thread. The thread has 130+ (not very interesting) entries in Deja, which is where I recovered this

Re: MA MCAS statistical fallacy

2001-01-13 Thread Herman Rubin
In article <93lc8j$fil$[EMAIL PROTECTED]>, Ronald Bloom <[EMAIL PROTECTED]> wrote: >Herman Rubin <[EMAIL PROTECTED]> wrote: >> In article <[EMAIL PROTECTED]>, >> J. Williams <[EMAIL PROTECTED]> wrote: >>>Francis Galton explained it in 1885. Possibly, the Mass. Dept. of >>>Education missed it! O

Re: MA MCAS statistical fallacy

2001-01-13 Thread Herman Rubin
In article <[EMAIL PROTECTED]>, Richard A. Beldin <[EMAIL PROTECTED]> wrote: >In one way or another, we have to give the slower students more time. We >can do it by making courses which allow students to progress (or not) at >their own pace or by flunking them so they can do it all over. The >for

Re: MA MCAS statistical fallacy

2001-01-13 Thread Herman Rubin
In article <[EMAIL PROTECTED]>, J. Williams <[EMAIL PROTECTED]> wrote: >Francis Galton explained it in 1885. Possibly, the Mass. Dept. of >Education missed it! Or, could it be that the same gang who brought >us the exit poll data during the November election were helping them >out? :-) >I am w

Re: AW: MA MCAS statistical fallacy

2001-01-13 Thread Gene Gallagher
In article <[EMAIL PROTECTED]>, <[EMAIL PROTECTED]> wrote: > Hi Dennis et al. > > The best reference concerning regression to the mean(rtm) is Dave Kenny! > Bookmark his homepage: > http://nw3.nai.net/~dakenny/kenny.htm > (Its an exciting one for still others reason than rtm ) > Dave had finaliz

AW: MA MCAS statistical fallacy

2001-01-12 Thread Werner Wittmann
TED] [mailto:[EMAIL PROTECTED]]Im Auftrag von dennis roberts Gesendet: Freitag, 12. Januar 2001 15:54 An: Gene Gallagher; [EMAIL PROTECTED] Betreff: Re: MA MCAS statistical fallacy At 01:12 PM 1/12/01 +, Gene Gallagher wrote: >I do believe that regression to the mean is involved here. i just re

Re: MA MCAS statistical fallacy

2001-01-12 Thread Gene Gallagher
In article <[EMAIL PROTECTED]>, [EMAIL PROTECTED] (Robert J. MacG. Dawson) wrote: > >The school results are presented in a very odd fashion, making it > >difficult to assess the patterns. > >http://www.doe.mass.edu/ata/ratings00/SPRPDistribTables.html > > They are that. Let's try. > > Thes

Re: MA MCAS statistical fallacy

2001-01-12 Thread Gene Gallagher
In article <[EMAIL PROTECTED]>, [EMAIL PROTECTED] (dennis roberts) wrote: > At 01:12 PM 1/12/01 +, Gene Gallagher wrote: > > >I do believe that regression to the mean is involved here. > > i just reiterate that regression in this case ... involves a correlation > between two columns of MEANS

Re: MA MCAS statistical fallacy

2001-01-12 Thread Robert J. MacG. Dawson
In my last posting I omitted the "very high" group on the grounds of small size. In case anybody's curious, here's what the plot looks like with those data included (coded as "*"; note that the vertical "error bars" on these would be very wide!) Proportion of schools improved by fewer

Re: MA MCAS statistical fallacy

2001-01-12 Thread Robert J. MacG. Dawson
>The school results are presented in a very odd fashion, making it >difficult to assess the patterns. >http://www.doe.mass.edu/ata/ratings00/SPRPDistribTables.html They are that. Let's try. These data don't look at all like the newspaper story. Here they are, with outcomes given as propo

Re: MA MCAS statistical fallacy

2001-01-12 Thread dennis roberts
At 01:30 PM 1/12/01 +, Gene Gallagher wrote: No, Each one of the 1539 schools in the state was evaluated. The districts were simply sent the results on the percent of their schools that had failed to meet their targets or that had met their target increases. All schools had to increase their

Re: MA MCAS statistical fallacy

2001-01-12 Thread dennis roberts
At 01:12 PM 1/12/01 +, Gene Gallagher wrote: >I do believe that regression to the mean is involved here. i just reiterate that regression in this case ... involves a correlation between two columns of MEANS ... means for schools OR means for districts ... and means do NOT change that much

Re: MA MCAS statistical fallacy

2001-01-12 Thread Gene Gallagher
In article <[EMAIL PROTECTED]>, [EMAIL PROTECTED] (dennis roberts) wrote: > i went to some of the sites given in the urls ... and, quite frankly, it is > kind of difficult to really get a feel for what has transpired ... and how > targets were set ... and how goals were assessed > > regardless o

Re: MA MCAS statistical fallacy

2001-01-12 Thread Gene Gallagher
In article <[EMAIL PROTECTED]>, [EMAIL PROTECTED] (dennis roberts) wrote: > > 1. how are categories of very high, etc. ... translated into 1 to 2 points > ... or 4 to 7 points? i don't see any particular connection of one to the other See the pdf link below. A panel set out cut points based on

Re: MA MCAS statistical fallacy

2001-01-12 Thread Eric Bohlman
Robert J. MacG. Dawson <[EMAIL PROTECTED]> wrote: > Therefore, I would not expect regression to the mean to be sufficient > to explain the observed outcome (in which "practically no" top schools > met expectations); and I conclude that the goals may well have been > otherwise unreasonable. I

Re: MA MCAS statistical fallacy

2001-01-12 Thread Gene Gallagher
In article <[EMAIL PROTECTED]>, [EMAIL PROTECTED] (Robert J. MacG. Dawson) wrote: > > > Wait one... Regression to the mean occurs because of the _random_ > component in the first measurement. Being in an urban center is not part > of the random component - those schools' grades didn't impr

Re: MA MCAS statistical fallacy

2001-01-12 Thread Richard A. Beldin
This is a multi-part message in MIME format. --A0316B6769484E5B80B17052 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit In one way or another, we have to give the slower students more time. We can do it by making courses which allow students to progress (or

Re: MA MCAS statistical fallacy

2001-01-11 Thread Ronald Bloom
Herman Rubin <[EMAIL PROTECTED]> wrote: > In article <[EMAIL PROTECTED]>, > J. Williams <[EMAIL PROTECTED]> wrote: >>Francis Galton explained it in 1885. Possibly, the Mass. Dept. of >>Education missed it! Or, could it be that the same gang who brought >>us the exit poll data during the November

Re: MA MCAS statistical fallacy

2001-01-11 Thread Ronald Bloom
Robert J. MacG. Dawson <[EMAIL PROTECTED]> wrote: > > (2) _Why_ were even the best schools expected to improve, with targets > that seem to have been overambitious?? I would hazard a guess that it > might be due to an inappropriate overgeneralization of the philosophy - > appropriate,

Re: MA MCAS statistical fallacy

2001-01-11 Thread Robert J. MacG. Dawson
Paul R Swank wrote: > > Robert: > > Why would you expect a strong correlation here? You're talking about tests done a >year apart with some new kids in each school and some kids who have moved on. Simply because there seems to be general consensus that there are such things as "good

Re: MA MCAS statistical fallacy

2001-01-11 Thread Rich Ulrich
On Wed, 10 Jan 2001 21:32:43 GMT, Gene Gallagher <[EMAIL PROTECTED]> wrote: > The Massachusetts Dept. of Education committed what appears to be a > howling statistical blunder yesterday. It would be funny if not for the > millions of dollars, thousands of hours of work, and thousands of > studen

Re: MA MCAS statistical fallacy

2001-01-11 Thread Paul R Swank
Robert: Why would you expect a strong correlation here? You're talking about tests done a year apart with some new kids in each school and some kids who have moved on. Is regression toward the mean causing all of the noted results. Probably not. But it is quite conceivable that it could be partial

Re: MA MCAS statistical fallacy

2001-01-11 Thread Herman Rubin
In article <[EMAIL PROTECTED]>, J. Williams <[EMAIL PROTECTED]> wrote: >Francis Galton explained it in 1885. Possibly, the Mass. Dept. of >Education missed it! Or, could it be that the same gang who brought >us the exit poll data during the November election were helping them >out? :-) >I am w

Re: MA MCAS statistical fallacy

2001-01-11 Thread Robert J. MacG. Dawson
Paul R Swank wrote: > > Regression toward the mean occurs when the pretest is used to form the groups, which >it appears is the case here. Of course it "occurs": - but remember that the magnitude depends on r^2. In the case where there is strong correlation between the pretest and the

Re: MA MCAS statistical fallacy

2001-01-11 Thread dennis roberts
i went to some of the sites given in the urls ... and, quite frankly, it is kind of difficult to really get a feel for what has transpired ... and how targets were set ... and how goals were assessed regardless of whether we like this kind of an approach for accountability ... or not ... we al

Re: MA MCAS statistical fallacy

2001-01-11 Thread dennis roberts
At 11:31 PM 1/10/01 -0500, Bob Hayden wrote: regression to the mean applies to relative position ... NOT raw scores let's say we give a test called a final exam at the beginning of a course ... and assume for a moment that there is some spread ... though the mean necessarily would be rather lo

Re: MA MCAS statistical fallacy

2001-01-11 Thread Paul R Swank
Regression toward the mean occurs when the pretest is used to form the groups, which it appears is the case here. At 08:31 AM 1/11/01 -0400, you wrote: > > >Gene Gallagher wrote: >> >> Those familiar with "regression to the mean" know what's coming next. >> The poor schools, many in urban centers

Re: MA MCAS statistical fallacy

2001-01-11 Thread J. Williams
Francis Galton explained it in 1885. Possibly, the Mass. Dept. of Education missed it! Or, could it be that the same gang who brought us the exit poll data during the November election were helping them out? :-) I am wondering why they did not have a set of objective standards for ALL student

Re: MA MCAS statistical fallacy

2001-01-11 Thread Robert J. MacG. Dawson
A couple additional thoghts I didn't get around to before leaving for my 8:30 lecture: (1) The clearest way of looking at the stats side of things is probably that one would expect a high enough r^2 between schools' performances in one year and in the next that regression to the m

Re: MA MCAS statistical fallacy

2001-01-11 Thread Robert J. MacG. Dawson
Gene Gallagher wrote: > > Those familiar with "regression to the mean" know what's coming next. > The poor schools, many in urban centers like Boston, met their ^ > improvement "targets," while most of the state's top school districts > failed

MA MCAS statistical fallacy

2001-01-10 Thread Bob Hayden
A powerful case for competency testing of all public officials!-) - Forwarded message from Gene Gallagher - The Massachusetts Dept. of Education committed what appears to be a howling statistical blunder yesterday. It would be funny if not for the millions of dollars, thousands of hour

MA MCAS statistical fallacy

2001-01-10 Thread Gene Gallagher
The Massachusetts Dept. of Education committed what appears to be a howling statistical blunder yesterday. It would be funny if not for the millions of dollars, thousands of hours of work, and thousands of students' lives that could be affected. Massachusetts has implemented a state-wide mandato