VOLTOLINI wrote:
>
> What is a P value?
>
> Searching in books I have found different definitions. The explanation for P
> = 0,79 could be.....
>
> 1 - the probability of Type I error.
> 2 - the probability of rejecting a correct null hypothesis.
> 3 - the probability of incorrectly rejecting the null hypothesis.
> 4 - the probability of rejecting a null hypothesis when in fact it is true.
All of these are wrong. These define (mostly sloppily - see below) the
"significance level" of a fixed level test. When you are reporting a
p-value as your final result no hypothesis is ever rejected. When you
compare a p-value to a fixed value "alpha" for a "reject/don't-reject"
test, it is the fixed value "alpha" that determines the "probability of
a type I error", not p.
The terminology "type I error" is very bad, because its definition
("rejecting the null hypothesis when it is true") can *only* be
understood as a macro that cuts across syntactic boundaries. Compare the
traditional C beginner's error:
#define FOO A+B
C = D * FOO
in which C evaluates to (D*A) + B not D*(A+B) as probably expected.
Similarly, it is too easy to read (1) above as
"the probability that (the null hypothesis is true and we reject it)"
when what is meant is
"the [conditional] probability that we reject the null hypothesis, given
that it is true".
Moreover, the term is often put into contexts such as "the probability
that a Type I error occurs" in which the real meaning is practically
unextractable. A Bayesian would say that in most situations, the
probability that "a Type I error occurs" is 0, because the probability
that the null hypothesis is exactly true is 0. A frequentist would say
it is undefined because the null hypothesis is not an "event".
I would say that (2) and (3) share this weakness; (4) is a nearly
acceptable definition of alpha (NOT p) in a fixed-level hypothesis test.
I would prefer
"The probability of rejecting a null hypothesis conditional on its
being true"
or
"The probability of rejecting a null hypothesis, given that it is
true".
> 6 - the strength of evidence against an hypothesis
No, no, no! This doesn't even point in the right direction. The larger
the p value the less evidence against the hypothesis.
> 5 - the chance of getting a test statistic as extreme or more extreme than this one
This is basically correct, if you add "assuming the null hypothesis to
be correct" (and say what you mean by "extreme")
> 7 - the probability of the observed difference between groups (or a larger
> difference) occurs
> 8 - the probability of obtaining a result equal to or more extreme than what
> was actually observed.
These are not quite correct. Both should have "for this sample size"
appended, (7) is only true for certain tests. and (8) does not explain
how "extremeness" is defined.
Another usable definition is "the smallest critical value alpha for
which we would reject the null hypothesis based on these data."
A p-value of 0.79 menas "essentially no evidence against the null
hypothesis." (This does not, of course, mean that the null hypothesis
is true - it may well be that the null hypothesis is false but by a
small enough amount that we need lots more data to show it!)
-Robert Dawson
.
.
=================================================================
Instructions for joining and leaving this list, remarks about the
problem of INAPPROPRIATE MESSAGES, and archives are available at:
. http://jse.stat.ncsu.edu/ .
=================================================================
