During the last week in August, there was a lengthy thread on sci.stat.edu
about problems with the probability and statistics questions in MCAS, the high
stakes test required for graduating from a MA public high school.
Shortly after participating in that thread, I wrote up my analyses of 6 of the
41 questions on the 2001 exam and 4 of the 42 questions on the 2000 exam. I
mentioned this group and several people who posted on that thread in my
commentary. Four of the 2000 & 2001 questions were in the area of statistics
and probability. The most severe problems appear to be testing on concepts
intended only for 11th & 12th graders - a violation of the basic principle of
MCAS. My commentary includes cross-references to these learning standards.
I sent my critique to the Dept. of Education in Aug. so that they could delete
the flawed questions from the scores on which failure decisions are based. As
described in the newspaper article linked below, there are no appeals now
possible in MCAS for failing because of invalid questions. Students must pass
this MCAS math test, or one of 4 retakes, to graduate in 2003. 45-50% of MA
10th graders failed the tests offered from 1998 through 2000.
The MA Dept of Education's Press spokesman, Heidi Perlman, is quoted in a
Berkshire Eagle story that DOE stands behind all of the questions I discussed:
"But the Department of Education dismisses the professor's analysis of the
test and stands by all its questions. In fact, the department changed its
scoring policy on the 10th-grade tests this year to safeguard against errors,
officials said. .. "Some of the questions could've been slightly better worded,
but in total none of them are invalid questions," said department spokeswoman
Heidi Perlman. She said it was reviewed by department staff, but officials did
not agree with Gallagher's analysis. "We don't believe his criticisms are
valid," Perlman said.
Ms Perlman's comments can can be read in the following news article from the
Berkshire Eagle, by Jennifer Fenn:
http://search.newschoice.com/ArchiveDisplay.asp?story=d:\index\newsarchive
s\ne\nebe\fpg\20011002\1122674_mcaserrors.txt&source=www%2Eberkshireeagle%
2Ecom&puid=2269&paper=Berkshire+Eagle+Online
One reporter told me that he wasn't sure how to handle my analysis of these
questions because DOE says the questions are all valid, and I say that they
aren't. He said that he's not a mathematician and can't judge who is correct.
I said publish the questions, and let the mathematically literate readers
respond.
I would appreciate what the members of this group think about the questions,
especially about the statistics and probability questions. I realize that my
28-p commentary is a bit much, but the issue is very important to the Class of
2003 and other citizens of MA.
The math/stats questions, with my analysis can be viewed at:
http://www.es.umb.edu/edg/MCAS/mcasproblems.pdf
Gene Gallagher
UMASS/Boston
=================================================================
Instructions for joining and leaving this list and remarks about
the problem of INAPPROPRIATE MESSAGES are available at
http://jse.stat.ncsu.edu/
=================================================================
