Michael Granaas wrote:
> Someone, I think it was on this thread, mentioned Abelson's book
> "Statistics as Principled Argument". In this book Abelson argues that
> individual studies simply provide pieces of evidence for or against a
> particular hypothesis. It is the accumulation of the evidence that allows
> us to make a conclusion. (My appologies to Abelson if I have
> misremembered his arguments.)
It is perfectly true that 'individual studies simply provide pieces of
evidence for or against a
particular hypothesis' - but it is equally true that multiple studies do the
same. Assuming the multiple studies show the same results, the evidence is of
course stronger - but it is still 'only' evidence.
One can legitimately draw a conclusion on one or several studies. One's
confidence (and the confidence of others!) in the conclusion depends on the
strength of the evidence. One well designed, well carried out study with clear
results provides strong evidence which may be enough to convince most people.
Several such studies which support each other provide even stronger evidence.
On the other hand, replications of poorly designed studies leading to unclear
results may give a little more evidence, but not enough to convince people.
In an individual study, the p-value(s) used is a measure of the strength of
the evidence provided by the study - BUT it is totally dependent on the
validity of the design of the study, the choice of variables, the selection of
the sample, the appropriateness of the models used to obtain the p-value. So
it is important, but certainly only one brick in the wall.
And of course treating 5% as some God-given rule of importance is ridiculous.
(It is nearly as bad as the N>30 'law' for treating a sample as 'large'.) But
it is a useful benchmark figure.
Regards,
Alan
--
Alan McLean (alan.buseco.monash.edu.au)
Department of Econometrics and Business Statistics
Monash University, Caulfield Campus, Melbourne
Tel: +61 03 9903 2102 Fax: +61 03 9903 2007
=================================================================
Instructions for joining and leaving this list and remarks about
the problem of INAPPROPRIATE MESSAGES are available at
http://jse.stat.ncsu.edu/
=================================================================
