On Fri, 20 Oct 2000, David Hardman wrote:
> And it's almost too obvious to be worth stating, but let's
> not forget the role of replication in science. You may get
> a p value of p < .0001, but if no-one else can replicate it
> then your result may well be a fluke. Of course, the
> failures to replicate may not be so easy to publish...!
This is exactly the point that I was going to add to Dennis's comments,
guess David saved me the trouble.
Unfortunately, I think that replication is probably one of the most
overlooked issues in the discussion of hypothesis testing etc. We
(frequently) teach, and certainly act, as if we can make a decision based
on the weight of a single research effort. When we behave as if a
scientific knowledge can be arrived at through a single study it is no
wonder that we have so much trouble with p-values.
In some disciplines we have a near absense of multi-experiment papers.
Admittedly publication pressures are a great problem here, but at least
some of these single study publications are fueled by the myth that you
can reach a scientific conclusion based on a single study.
I don't know what to do about this, but I do know that I have changed my
teaching so as to encourage students to think about a study as being a
piece of the puzzle, not the solution to the entire puzzle. Ultimately
new findings need to be replicated under a variety of circumstances to
validate any new knowledge.
Someone, I think it was on this thread, mentioned Abelson's book
"Statistics as Principled Argument". In this book Abelson argues that
individual studies simply provide pieces of evidence for or against a
particular hypothesis. It is the accumulation of the evidence that allows
us to make a conclusion. (My appologies to Abelson if I have
misremembered his arguments.)
Michael
>
>
> Dr. David Hardman
>
> "Rational - Devoid of all delusions save those
> of observation, experience and reflection."
> - Ambrose Bierce (The Devil's Dictionary)
>
> Department of Psychology
> London Guildhall University
> Calcutta House
> Old Castle Street
> London E1 7NT
>
> Phone: +44 020 73201256
> Fax: +44 020 73201236
> E-mail: [EMAIL PROTECTED]
> Internet: http://www.lgu.ac.uk/psychology/hardman.html
>
> For information on the London Judgment and Decision Making Group
> visit: http://www.lgu.ac.uk/psychology/hardman/ljdm.html
>
> For information on joining the 'Decision' mailbase list, see
> http://www.mailbase.ac.uk/lists/decision
>
>
>
> =================================================================
> Instructions for joining and leaving this list and remarks about
> the problem of INAPPROPRIATE MESSAGES are available at
> http://jse.stat.ncsu.edu/
> =================================================================
>
*******************************************************************
Michael M. Granaas
Associate Professor [EMAIL PROTECTED]
Department of Psychology
University of South Dakota Phone: (605) 677-5295
Vermillion, SD 57069 FAX: (605) 677-6604
*******************************************************************
All views expressed are those of the author and do not necessarily
reflect those of the University of South Dakota, or the South
Dakota Board of Regents.
=================================================================
Instructions for joining and leaving this list and remarks about
the problem of INAPPROPRIATE MESSAGES are available at
http://jse.stat.ncsu.edu/
=================================================================
