On 30 Jan 2001, Will Hopkins wrote:
---------------------- >8 -----------------------
> I haven't followed this thread closely, but I would like to state the
> only valid and useful interpretation of the p value that I know. If
> you observe a positive effect, then p/2 is the probability that the
> true value of the effect is negative. Equivalently, 1-p/2 is the
> probability that the true value is positive.
>
> The probability that the null hypothesis is true is exactly 0. The
> probability that it is false is exactly 1.
Suppose you were conducting a test with someone who claimed to have ESP,
such that they were able to predict accurately which card would be turned
up next from a well-shuffled deck of cards. The null hypothesis, I think,
would be that the person does not have ESP. Is this null false?
And what about when one has a one-tailed alternative hypothesis, e.g., mu
> 100. In this case, the null covers a whole range of values (mu < or =
100). Is this null false? In such a case, one still uses the point null
(mu = 100) for testing, because it is the most extreme case. If you can
reject the point null of mu=100, you will certainly be able to reject the
null if mu is actually some value less than 100. But the point is, the
null can be true.
With a two-tailed alternative, the point null may not be true, but as one
of the regulars in these newsgroups often points out, we don't know the
direction of the difference. So again, it makes sense to use the point
null for testing purposes.
> Estimation is the name of the game. Hypothesis testing belongs in
> another century--the 20th. Unless, that is, you base hypotheses not
> on the null effect but on trivial effects...
Bob Frick has a paper with some interesting comments on this in the
context of experimental psychology. In that context, he argues, models
that make "ordinal" predictions are more useful than ones that try to
estimate effect sizes, and certainly more generalizable. (An ordinal
prediction is something like performance will be impaired in condtion B
relative to condition A. Impairment might be indicated by slower
responding and more errors, for example.)
A lot of cognitive psychologists use reaction time as their primary DV.
But note that they are NOT primarily interested in explaining all (or as
much as they can) of the variation in reaction time. RT is just a tool
they use to make inferences about some underlying construct that really
interests them. Usually, they are trying to test some theory which leads
them to expect slower responding in one condition relative to another, for
example--such as slower responding when distractors are present compared
to when only a target item appears. The difference between these
conditions almost certainly will explain next to none of the overall
variation in RT, so eta-squared and omega-squared measures will not be
very impressive looking. But that's fine, because the whole point is to
test the ordinal prediction of the theory--not to explain all of the
variation in RT. If one was able to measure the underlying construct
directly, THEN it might make some sense to try estimating parameters. But
with indirect measurements like RT, I think Frick's recommended approach
is a better one.
There's my two cents.
--
Bruce Weaver
New e-mail: [EMAIL PROTECTED] (formerly [EMAIL PROTECTED])
Homepage: http://www.angelfire.com/wv/bwhomedir/
=================================================================
Instructions for joining and leaving this list and remarks about
the problem of INAPPROPRIATE MESSAGES are available at
http://jse.stat.ncsu.edu/
=================================================================
