On Mon, 19 Feb 2001 04:27:24 GMT, [EMAIL PROTECTED] (Irving
Scheffe) wrote:
> In responding to Rich, I'll intersperse selected comments with
> selected portions of his text and append his entire post below.
- I'm not done with the topic yet. But it is difficult to go on from
this point.
I think the difficulty is that JS has constructed his straw-man
argument about how "hypotheses" are handled; and since it
is a stupid strategy, it is easy for him to claim that it is fatally
flawed.
>From his insistence on his "examples," it seems to me that he
believes that someone else is committed to using p-levels in a strict
way, by beating 5%. That's certainly not the case for me, and I
doubt if anyone defends or promotes it, outside of carefully designed
Controlled Random Experiments.
Despite the fact that I could not make sense of WHY he wanted
his example, it turns out -- after he explains it more -- that my own
analysis covered the relevant bases. I agree, if you don't have
"statistical power," then you don't ask for a 5% test, or (maybe)
any test at all. The JUSTIFICATION for having a test on the MIT
data is that the power is sufficient to say something.
And what it said is that Jim did BAD INFERENCE. I said that a
couple of times. I regret that I may have confused people with
unnecessary words about "inference."
Outlier => No central tendency => Mean is BAD statistic;
careful reader insists on more or better information before asserting
there's a difference.
I asserted that more than once.
Optimistically, my own data analysis technique might be described as
"starting out with everything Jim might figure out and conclude from
the data, and adding to that, flexible comparisons from related
fields, and other statistical tools." -- It was quite annoying for
me to read where he implicitly says, "You, idiot, would HAVE to
decide otherwise." I mean, I thought I wrote a lot clearer than that.
Now, below is a quotation that describes Jim's justifications, I
hope, in more detail than Jim does. This is from web site which I
just discovered, but which looks quite admirable -- except for this
question of "Sampling".
I think that Garson is wrong, and the last 40 years of epidemiological
research have proven the worth of statistics provided on non-random,
"observational" samples. When handled with care.
================
>From G. David Garson, "PA 765 Notes: An Online Textbook."
On Sampling
http://www2.chass.ncsu.edu/garson/pa765/sampling.htm
Significance testing is only appropriate for random samples.
Random sampling is assumed for inferential statistics
(significance testing). "Inferential" refers to the fact
that conclusions are drawn about relationships in the data
based on inference from knowledge of the sampling
distribution. Significance tests are based on a sampling
theory which requires that every case have a chance of being
selected known in advance of sample selection, usually an
equal chance. Statistical inference assesses the
significance of estimates made using random samples. For
enumerations and censuses, such inference is not needed
since estimates are exact. Sampling error is irrelevant and
therefore inferential statistics dealing with sampling error
are irrelevant. Significance tests are sometimes applied
arbitrarily to non-random samples but there is no existing
method of assessing the validity of such estimates, though
analysis of non-response may shed some light. The following
is typical of a disclaimer footnote in research based on a
non random sample:
"Because some authors (ex., Oakes, 1986) note the use of
inferential statistics is warranted for nonprobability
samples if the sample seems to represent the population, and
in deference to the widespread social science practice of
reporting significance levels for nonprobability samples as
a convenient if arbitrary assessment criterion, significance
levels have been reported in the tables included in this
article." See Michael Oakes (1986). Statistical inference: A
commentary for social and behavioral sciences. NY: Wiley.
================
Maybe we can pick up the discussion from here?
--
Rich Ulrich, [EMAIL PROTECTED]
http://www.pitt.edu/~wpilib/index.html
=================================================================
Instructions for joining and leaving this list and remarks about
the problem of INAPPROPRIATE MESSAGES are available at
http://jse.stat.ncsu.edu/
=================================================================
