http://askearth.com/go/view_request?request=5058
=
Instructions for joining and leaving this list, remarks about the
problem of INAPPROPRIATE MESSAGES, and archives are available at
http://jse.stat.ncsu.edu/
Dr. Gallagher,
I'm pleased that my post about the TI-83 was helpful. Your postings
(and those of others) on the problem with MCAS have been both
enlightening and disturbing. I'm going to use the MCAS as an
example of what can go wrong with a test. Also, this discussion has
inspired me
Dr. Mecklin,
Thanks so much for your post about the TI83 calculator. On a hunch, I looked at the MCAS instructions page. The boxplot problem was asked in Section 3 of the 3-d test, and in the final 2 sections, students were encouraged to bring their graphing calculators. The MCAS test writers
: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]
Sent: Thursday, October 04, 2001 9:33 PM
To: [EMAIL PROTECTED]
Subject: MCAS, statistics and other math problems
During the last week in August, there was a lengthy thread on
sci.stat.edu
about problems with the probability and statistics
At 07:03 AM 10/5/01 -0500, Olsen, Chris wrote:
Professor Gallagher and All --
It would appear that neither the appeal systems nor a claim of
technical adequacy would be a response to your concern about bad
questions. The claim of technical adequacy, i.e. that good students tend
to answer
Thanks, and moral support is appreciated.
MCAS is very controversial in MA. None of the major papers have yet called
into question the test itself. Now, the DOE apparently is saying that they
stand behind every question.
I believe it might help if the MA DOE MCAS group, headed by Jeff
Dr. Gallagher and Edstat newsgroup:
Here's my take on the MCAS and boxplots.
(1) I agree with Eugene Gallagher and others in that the question about
boxplots on the MCAS is poor, since the correct answer depends on whether
you learned the Tukey boxplot (that indicates outliers) or the quick
At 12:41 PM 10/5/01 -0500, Christopher J. Mecklin wrote:
(4) If the Massachusetts Department of Education really wants to include a
boxplot item on the test, it should either be a multiple choice question
written so that the correct answer is the same no matter which type of
boxplot one was
During the last week in August, there was a lengthy thread on sci.stat.edu
about problems with the probability and statistics questions in MCAS, the high
stakes test required for graduating from a MA public high school.
Shortly after participating in that thread, I wrote up my analyses of 6
IMHO, no you are not being an idiot (I'll leave question of 'again' up to you
:)
the question of what is an outlier, and what is not, comes up every time you
have data collected by undergraduates in a standard lab. They believe the data
should fit some pre-arranged principle/equation. So
On 30 Aug 2001 12:14:21 -0700, [EMAIL PROTECTED] (Jill Binker)
wrote:
[ snip, prior post ]
Well, yes. You should always stare at all your data (though I guess a lot
of people leave this crucial step out). I thought there were two important
reasons to look for outliers.
An outlier may be a
On Tue, 28 Aug 2001, Dennis Roberts wrote in part:
however ... the flagging of outliers is totally arbitrary ... i
see no rationale for saying that if a data point is 1.5 IQRs away from
some point ... that there is something significant about that
If the data are normally distributed (or
I wrote:
Er, no.
Q1 ~ mu - 2/3 sigma
Q3 ~ mu + 2/3 sigma
1 IQR ~ 4/3 sigma
1.5 IQR ~ 2 sigma
inner fence ~ mu +- 2 2/3 sigma which is about the 0.5 percentile.
-right so far -
and then burbled
The inner fences are selected
boxplot which uses the
box +/- 1.5 IQR rule. (Of course, if there are no outliers -- by that
definition -- the two are identical.)
Interesting. As noted in earlier posts, the National Council of Mathematics
Teachers and MCAS include only the quick boxplot in their definition of
boxplot. The 10th
At 9:41 AM -0400 8/30/01, Dennis Roberts wrote:
all of this is assuming of course, that some extreme value ... by ANY
definition ... is bad in some way ... that is, worthy of special
attention for fear that it got there by some nefarious method
i am not sure the flagging of extreme values has
consequence.
So long as the questions have the same relationship to technical content as
the famous MCAS #39 has to what we call statistical thinking (and doing), we
can expect more and continuing distortions of education content. What teacher
would struggle to understand and communicate the real thing
I wrote:
An obvious approach that would seem to give the advantages hoped for
from the focussed test without the disadvantages would be just to group
questions in the original test in roughly increasing order of
difficulty.
which, I think, answers Dennis' question.
I
(and the MCAS test writers) have
adopted
one feature of the Tukey boxplot, but not the most important feature: the
ability to flag outliers.
Eugene Gallagher
however ... the flagging of outliers is totally arbitrary ... i see no
rationale for saying that if a data point is 1.5 IQRs away from
At 01:33 PM 8/28/01 -0500, Jay Warner wrote:
Suggest we step back a minute.
by de facto definition ... the MCAS tests ... are intended to convey ...
MINIMUM skills/knowledge that they expect all high school GRADUATES to have
... they certainly cannot purport to test and/or represent anything
I got an email from Anand Vaishnav, the Globe reporter who did Friday's article
on the math and stats problems in MCAS. Only about 50% of the 63000 10th
graders in MA got median and range. I suspect that mean and range probably was
the most popular incorrect answer (according to the MCAS
Dennis Roberts [EMAIL PROTECTED] wrote:
if we take the infamous #39 item ... where the options were (if i recall)...
A. mean only
B. median only
C. range and mean
D. range and median
well, even if we accepted this item as fair ...
a student looks at the graph ... sees that there is a
Eric Bohlman wrote:
And furthermore, not all the wrong answers are equally bad. Someone who
would answer A or B must know quite a bit less than someone who would
answer C (in fact, it would tend to indicate that they had no concept at
all of what the boxplot represented).
I don't believe
At 10:43 PM 8/28/01 +, EugeneGall wrote:
I got an email from Anand Vaishnav, the Globe reporter who did Friday's
article
on the math and stats problems in MCAS. Only about 50% of the 63000 10th
graders in MA got median and range. I suspect that mean and range
probably was
the most popular
At 11:30 PM 8/28/01 +, Jim Callahan wrote:
Eric Bohlman wrote:
And furthermore, not all the wrong answers are equally bad. Someone who
would answer A or B must know quite a bit less than someone who would
answer C (in fact, it would tend to indicate that they had no concept at
all of
necessary to survive at the next level. In the case
of the MCAS tests, they must pass it to graduate from high school. Asking
questions that are at the high end of the expected learning level (don't ask me
for an operational definition of that - I don't make these terms up:) is a
waste of time - we
David Winsemius wrote
Here are the three stated competencies that the test is supposed to measure:
http://www.doe.mass.edu/mcas/01release/
1) Select, create, and interpret an appropriate graphical representation
(e.g.,
scatterplot, table, stem-and-leaf plots, box-and-whisker plots, circle
graph
Jay Warner wrote:
Trusting that Eugene or someone in Massachusetts can respond, I ask,
Are box and whisker plots included in the material that _must_ be taught
in the
K-12 curriculum? I believe a previous post pointed out that older intro stat
books, including those listed as suitable for MCAS
MCAS test is a make-or-break
high stakes test. A straight-A average won't get you a diploma if you fail
this test, and the majority of 10th graders in MA fail this test, as it is now
being scored. 80% of hispanic students, 76% of black students, and 36% of
white students in MA failed the 2000 10th
At 12:31 PM 8/27/01 +, EugeneGall wrote:
The Harcourt-Brace description of the boxplot, which is now being taught
to MA students, isn't a proper boxplot (maybe the Harcourt-Brace K-12 boxplot
is different from the Tukey boxplot), becuase it doesn't properly plot
outliers
and extreme
/
Silber abstract:
Ed board goes off course
Boston Herald; Feb 2, 2001;
by John SILBER;
The board also authorized two new tests in addition to the standard MCAS test:
the
parallel test and the focused test. These terms are typical educationist
smog. As far as one can make out from the Department
On 27 Aug 2001 12:11:12 GMT, [EMAIL PROTECTED] (EugeneGall)
wrote:
David Winsemius wrote
Here are the three stated competencies that the test is supposed to measure:
http://www.doe.mass.edu/mcas/01release/
I think it is legitimate for a class-room exam to offer
an item that rewards classroom
Dennis Roberts wrote:
At 12:31 PM 8/27/01 +, EugeneGall wrote:
The Harcourt-Brace description of the boxplot, which is now being taught
to MA students, isn't a proper boxplot (maybe the Harcourt-Brace K-12
boxplot
is different from the Tukey boxplot), becuase it doesn't properly plot
The focused test eliminates questions that would enable students to score A's,
B's, and C's. All they get is another chance to score a D rather than an F.
Implicit in the very concept of the focused test is the idea that a student who
fails the standard MCAS test cannot be more than a D
At 01:57 PM 8/27/01 -0300, Robert J. MacG. Dawson wrote:
The focussed test isn't an entirely bad idea; it does allow a
genuine D
student to avoid getting blown out of the water by questions intended to
discriminate between A and B students. However, it seems like a very
poor way to
Robert Dawson wrote:
An obvious approach that would seem to give the advantages hoped for
from the focussed test without the disadvantages would be just to group
questions in the original test in roughly increasing order of
difficulty. One might (I'm not so sure that this would be a good
I just took another look at the front page Boston Globe article that described
the most difficult MCAS question. The most difficult question on the 2001 math
test is a question involving probability and geometry, question 9 on the MCAS
test
http://www.doe.mass.edu/mcas/01release/
I've seen
gt; fails the standard MCAS test cannot be more than a D student. Such
students
> will be accommodated with a special test on which they cannot score
better than
> a D: MCAS for Dummies. The cynicism of the proposal is almost beyond
belief ...
> The board is telling every 10th-grader in Mas
serves is to test those
things that are a part of the state curriculum in Massachusetts. If the
MCAS is testing material not on the curriculum, this would also seem to be a
much more serious problem than median vs. mean for a boxplot.
-- Chris
Chris Olsen
George Washington High School
2205
At 10:40 AM 8/26/01 -0500, Olsen, Chris wrote:
Dear Sir or Madam:
I read with interest your posting on the issue of the so-called Tukey
boxplot. I would like to make a few observations, if you will forgive the
temerity of a high school teacher.
since i was not the person posting the
involved.
I am not familiar with the MCAS test, nor for that matter, the curriculum in
Massachusetts. However, I can imagine a public uproar if a high stakes test
delivered low percentages of correct scores for a significant part of the
population, unless there was a very low cut score for passing
can be downloaded as a pdf at:
http://www.doe.mass.edu/mcas/01release/
The exam shows a box and whisker plot.
The boxplot is question 39 of 40 The other math/stats questions are 9, 26, 36,
37 and 40.
I take back my earlier criticism of this question. The question is worse than
I thought
the first hurdle an item has to pass is ... a content one ... is the
content that we are asking about ... in the item ... sensible ... important
enough ... to spend 1 of the 6 items worth on the test ... given all the
concepts that could be tested on the test
if the answer to this is yes ...
Which statisticsmean, median, mode,
rangecan be determined from this
graph?
A. mean only
B. median only
C. range and mean
D. range and median
.
here is the item ... without the boxplot ... now, the lower point
is roughly 15 ... the spot above the | is about 26 ... and the upper end
point is
Since there were six questions regarding statistics on the test, I'd like to
see what questions those members of the mailing list might propose rather
than the six that were used.
Dr. Robert C. Knodt
4949 Samish Way, #31
Bellingham, WA 98226
[EMAIL PROTECTED]
The man who does not read good
Here are the three stated competencies that the test is supposed to measure:
http://www.doe.mass.edu/mcas/01release/
1) Select, create, and interpret an appropriate graphical representation
(e.g.,
scatterplot, table, stem-and-leaf plots, box-and-whisker plots, circle
graph, line
graph, and line
EugeneGall wrote:
[snip]
I'm really curious why the Boston Globe did not publish this question and
answer. Many suspect that the Globe is going easy on MCAS. Could it be that
the Globe consultants realized this 2001 MCAS boxplot question was a stinker
for such an important make-or-break
Some of the MCAS stats and probability questions were tough but fair. I
disagreed vehemently with one question:
Question 39 on the 10th grade Math 2001 test.
It showed a Tukey boxplot and asked whether the graph represented a mean and
range or a median and range.
Now, this question will do one
... and if someone misses it ... they should not be docked as much as many
other more fundamentally important items ... that hopefully WILL be put on
the test
At 02:17 PM 8/25/01 +, EugeneGall wrote:
Some of the MCAS stats and probability questions were tough but fair. I
disagreed vehemently
In a message dated 8/25/01 9:06:18 AM Pacific Daylight Time, [EMAIL PROTECTED]
writes:
whether the item you talk about rises to the level of being important
enough ... i am not sure ... certainly, in the overall scheme of things ...
IF it is included ... it would have to be considered to
) The variance of the sampe
3) The standard deviation of the sample
4) Absolutely nothing
Of course, the answer is 4) but students/parents and teachers would really be
outraged at this question and answer. This question is every bit as valid as
the MCAS question 39. The answer hinges only on a trivial
conceptually yes ... since, some items are clearly more important than
others ...
empirically ... since this has been explored many times ... back in the 70s
and 80s ... it seems to have no impact on rel and val ...
who decides? well, one way is to say ... experts ... have them rate items
in
At 07:35 PM 8/25/01 +, EugeneGall wrote:
The answer hinges only on a trivial choice made by Tukey
when he described the boxplot. Incidentally, Tufte criticized the lack of
information in the box width in the Tukey boxplot and proposed an alternative.
i don't think this is relevant to the
/MCAS_results_show_weakness_in_teens_grasp_of_probabilityP.shtml
MCAS results show weakness in teens' grasp of probability
By Anand Vaishnav, Globe Staff, 8/24/2001
ome Massachusetts math teachers say they take a gamble and don't spend
too much class time teaching probability, and it apparently showed on the
2001 MCAS exam
On 22 Jan 2001 15:58:16 -0800, [EMAIL PROTECTED] (dennis roberts) wrote:
At 03:28 PM 1/22/01 -0500, Rich Ulrich wrote:
snip, details of my alternative examples of statements
as i said before ... given the stem and the choice C of 1 foot ... i think
any intelligent examinee could argue
i think that one thing that math class teaches you about "measurement" is
that there is error ... well maybe they do, now that i think of it, i am
not so sure how clear this notion is taught ... but, let's assume that it
is ...
math would also reinforce meaning of the use of the english (
for the
1999 exam:
http://www.doe.mass.edu/mcas/99results/interp99/full_guide.pdf
...
In other DOE documents it states that a score over 280 or less than 200
is possible on some tests in some years, but these extremes will be
converted to 200 or 280.
In the following technical report
dennis roberts wrote:
well, if george bush pushes for more testing for accountability ... across
the nation like in texas, then i suspect more WILL get into this debate ...
either voluntarily or, being forced into it for some reason or another
the major problem is not that the community of
Gene Gallagher wrote:
[snip]
I have no idea what logic went into developing this 200 - 280 point
scaling system. The point system for grading Las Vegas boxing matches
makes as much sense to me
[snip again] Check out page 33 in the pdf and 31 in the report. There were 72 points
Jay and All --
I have been following with interest the discussion on MCAS.
Jay Warner writes...
the idea of measuring educational performance is fundamentally
worthwhile to meet certain objectives. the methods described
here fall far short of any ideal,and appear to fall short
There is another problem with testing, bureaucracies, schools, etc. One
does not "teach to the test," rather one "teaches to the budget."
Individual teachers may be rewarded by how well their students do on the
tests and so teach the tests. The school may be in the situation that a
poor school
Gene Gallagher wrote:
The scale is set from 200 to 280, but the maximum score that I've seen
so far is 257 (Boston Latin Math). The DOE only provides these data as
pdf's so it is difficult to find the max.
OK, so I didn't dream it. That leaves the question:
dennis roberts wrote:
there just is no good way to argue against the original choice C ... IN THE
CONTEXT OF THE STEM OF THE QUESTION
I am reminded of the joke article that contains many `pollitically incorrect'
answers to the exam question "given a barometer how do you measure
the hight of a
"P.G.Hamer" wrote:
dennis roberts wrote:
there just is no good way to argue against the original choice C ... IN THE
CONTEXT OF THE STEM OF THE QUESTION
I am reminded of the joke article that contains many `politically incorrect'
answers to the exam question "given a barometer how
"P.G.Hamer" wrote:
dennis roberts wrote:
there just is no good way to argue against the original choice C ... IN THE
CONTEXT OF THE STEM OF THE QUESTION
I am reminded of the joke article that contains many `pollitically incorrect'
answers to the exam question "given a barometer how do
while gene has read these documents carefully (someone has to right?) ... i
have not BUT, the following tidbit struck my interest
Comparisons of School- and District-Level Scores
The statistical significance of these comparisons is based on the number of
students tested. Whether
"Daniel P. B. Smith" wrote:
Can anybody possibly believe that a difference of one point in 245.3 can
possibly be significant? We're talking about schools with a less than a
maybe sixty fourth-graders in them. This just runs against common
sense...
Didn't somebody say a week or
In article [EMAIL PROTECTED],
[EMAIL PROTECTED] (Robert J. MacG. Dawson) wrote:
"Daniel P. B. Smith" wrote:
Can anybody possibly believe that a difference of one point in 245.3
can
possibly be significant? We're talking about schools with a less
than a
maybe sixty fourth-graders in
Callahan 245.3-1.0
Oldham242.2 6.5
Prescott 242.5 5.5
These seem to me like minuscule differences.
Needless to say, all public reporting and discussion of MCAS scores
seems to assume that the scores are perfectly accurate, with no stated
margin
On Tue, 16 Jan 2001 14:14:36 GMT, [EMAIL PROTECTED] (J. Williams)
concluded:
Maybe, I am missing something, but think the original question and
response items are quite clear and concise. I see nothing
particularly "loose" about it. The essentials of a class interval
used in frequency
At 03:57 PM 1/17/01 -0500, Rich Ulrich wrote:
- Okay, here is my answer before I repeat the official ones.
The "greatest possible difference" is *at least* one foot.
If this is a dedicated math question, the aspect of roundoff should
give "one foot (minimum)"; and any slightest introduction
On Mon, 15 Jan 2001 19:47:49 -0500, Rich Ulrich [EMAIL PROTECTED]
wrote:
Concerning the MCAS. There was a discussion last month
in another Usenet group, alt.usage.english, concerning one of
its math questions which was written too loosely.
Here is the start of that thread. The thread has
Rich Ulrich wrote:
Construing the language as precisely as possible, but being careful to
take into account the full language of the question and the
multiple-choice answers, what do you think the correct answer is?
Do you think the question is actually OK? Is the wording good enough
as
Werner Wittmann wrote:
See the regression artifact primer at Dave's homepage:
http://nw3.nai.net/~dakenny/rrtm.htm
I looked at
http://nw3.nai.net/~dakenny/primer.htm
and found myself puzzled by the Galton squeeze plot (or is it a pair
link diagram, or are they one and the same?).
It shows
At 11:33 AM 1/16/01 -0400, you wrote:
37. When Matt's and Damien's broad jumps were measured accurately to
the nearest foot, each measurement was 21 feet. Which statement best
describes the greatest possible difference in the lengths of Matts
jump and Damien's jump?
A. One jump could be
In article [EMAIL PROTECTED],
J. Williams [EMAIL PROTECTED] wrote:
Francis Galton explained it in 1885. Possibly, the Mass. Dept. of
Education missed it! Or, could it be that the same gang who brought
us the exit poll data during the November election were helping them
out? :-)
I am wondering
In article [EMAIL PROTECTED],
Richard A. Beldin [EMAIL PROTECTED] wrote:
In one way or another, we have to give the slower students more time. We
can do it by making courses which allow students to progress (or not) at
their own pace or by flunking them so they can do it all over. The
former is
Alan Zaslavsky wrote:
There is a big statistical literature on how to measure school performance.
The models used tend to be relatively complex ones, e.g., multilevel models to
take account of region, school, year and so on. The sad thing here is that
(whatever the main cause or causes) they are
This is a multi-part message in MIME format.
--A0316B6769484E5B80B17052
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
In one way or another, we have to give the slower students more time. We
can do it by making courses which allow students to progress (or
was hoping for (a mean 2 point improvement). Rather
than reporting that the mean had improved, this Dept. of Education
report emphasizes failure. Schools in this high category were expected
to improve by 2 points on the 200 to 280 point MCAS scale. A high
performing school might have an overall sc
Robert J. MacG. Dawson [EMAIL PROTECTED] wrote:
Therefore, I would not expect regression to the mean to be sufficient
to explain the observed outcome (in which "practically no" top schools
met expectations); and I conclude that the goals may well have been
otherwise unreasonable.
it over one that they preferred because the content
was closer to that being tested with the MCAS. He said that their
previous model was that earth sciences were dealt with in a unified
package in one year, followed by the life sciences in the years before
and after. However, the MCAS tests both
The school results are presented in a very odd fashion, making it
difficult to assess the patterns.
http://www.doe.mass.edu/ata/ratings00/SPRPDistribTables.html
They are that. Let's try.
These data don't look at all like the newspaper story. Here they are,
with outcomes given as
In my last posting I omitted the "very high" group on the grounds of
small size. In case anybody's curious, here's what the plot looks like
with those data included (coded as "*"; note that the vertical "error
bars" on these would be very wide!)
Proportion of schools improved by fewer
At 11:42 AM 1/12/01 -0600, Paul R Swank wrote:
snip
The local schools are already being forced to teach to the test.
AH HA ... THE TEXAS PLAN
=
Instructions for joining and leaving this list and remarks about
the problem of
In article [EMAIL PROTECTED],
[EMAIL PROTECTED] (Robert J. MacG. Dawson) wrote:
The school results are presented in a very odd fashion, making it
difficult to assess the patterns.
http://www.doe.mass.edu/ata/ratings00/SPRPDistribTables.html
They are that. Let's try.
These data
von dennis roberts
Gesendet: Freitag, 12. Januar 2001 15:54
An: Gene Gallagher; [EMAIL PROTECTED]
Betreff: Re: MA MCAS statistical fallacy
At 01:12 PM 1/12/01 +, Gene Gallagher wrote:
I do believe that regression to the mean is involved here.
i just reiterate that regression in this case
Gene Gallagher wrote:
Those familiar with "regression to the mean" know what's coming next.
The poor schools, many in urban centers like Boston, met their
^
improvement "targets," while most of the state's top school districts
failed to
A couple additional thoghts I didn't get around to before leaving for
my 8:30 lecture:
(1) The clearest way of looking at the stats side of things is probably
that one would expect a high enough r^2 between schools' performances in
one year and in the next that regression to the
for the
millions of dollars, thousands of hours of work, and thousands of
students' lives that could be affected.
Massachusetts has implemented a state-wide mandatory student testing
program, called the MCAS. Students in the 4th, 8th and 10th grades are
being tested and next year 12th grade students
Regression toward the mean occurs when the pretest is used to form the groups, which it appears is the case here.
At 08:31 AM 1/11/01 -0400, you wrote:
>
>
>Gene Gallagher wrote:
>>
>> Those familiar with "regression to the mean" know what's coming next.
>> The poor schools, many in urban
...
then regression to the mean is not the bugaboo that the "letter" makes it
out to be
now, the post said:
The effectiveness of school districts is being assessed using average
student MCAS scores. Based on the 1998 MCAS scores, districts were
placed in one of 6 categories: very high, high, moderate,
i went to some of the sites given in the urls ... and, quite frankly, it is
kind of difficult to really get a feel for what has transpired ... and how
targets were set ... and how goals were assessed
regardless of whether we like this kind of an approach for accountability
... or not ... we
Dear Gene,
I share your feelings about MCAS (fortunately my daughter finished high
school before it came into effect, but that's no consolation for the
thousands of other kids who are supposed to be deprived of their high
school diplomas), and had some similar reactions to the article
Robert:
Why would you expect a strong correlation here? You're talking about tests done a year apart with some new kids in each school and some kids who have moved on.
Is regression toward the mean causing all of the noted results. Probably not. But it is quite conceivable that it could be
met by becoming only a little worse. If you are
dumping money into the whole system, then you might
hope to (expect to?) bias the changes into a positive direction.
I thought it was curious that the
"schools in the highest two categories were expected to increase
their average MCAS score
Herman Rubin [EMAIL PROTECTED] wrote:
In article [EMAIL PROTECTED],
J. Williams [EMAIL PROTECTED] wrote:
Francis Galton explained it in 1885. Possibly, the Mass. Dept. of
Education missed it! Or, could it be that the same gang who brought
us the exit poll data during the November election
mandatory student testing
program, called the MCAS. Students in the 4th, 8th and 10th grades are
being tested and next year 12th grade students must pass the MCAS to
graduate.
The effectiveness of school districts is being assessed using average
student MCAS scores. Based on the 1998 MCAS scores
97 matches
Mail list logo
