At 10:30 AM 5/15/00 -0500, Simon, Steve, PhD wrote:
>There have been a lot of interesting comments in this thread. Let me just
>add my two cents.
>
>Anyway, what I tell them is that nine times out of ten, the mistake was not
>in how the data was analyzed, but in how it was collected. After all, if you
>collect the wrong data, it doesn't matter how sophisticated the analysis is,
>does it?
many MANY years ago ... early in the 80s ... i presented a little paper at
a little conference at a little school titled: THE OVERRATED IMPORTANCE OF
STATISTICS IN RESEARCH ... where i took a little 2 group
experimental/control group design ... and listed some steps in this process
such as:
defining TARGET population
taking a sample FROM that population
subdivision OF sample into exp and cont groups
IMPLEMENTATION of the treatment
use of RELIABLE measures
using APPROPRIATE analyses
engaging in REASONABLE interpretations of the data
and other things ...
and tried to show that in the entire scheme of things ... that the process
of using appropriate statistical analysis was the LEAST important of the
batch ... and in fact, rather paled in importance when compared to the mess
one can easily get into when one or more of the OTHER 'steps' has some flaw
... sometimes potentially fatal
recovery from some INappropriate analysis is accomplished rather easily
but, recovery from mistakes in the other areas is usually almost impossible
to do ...
i sometimes have a relook at this paper ... especially when we tend to get
so hung up in trivial details of various analysis methods ... and
especially when we try to make a big deal out of something like a p value
... for some specific test ... when i bet a quarter to a penny that there
are so many other problems within the context of 'typical' studies as to
make such p value squabbles rather silly ...
Dennis Roberts, EdPsy, Penn State University
208 Cedar Bldg., University Park PA 16802
Email: [EMAIL PROTECTED], AC 814-863-2401, FAX 814-863-1002
WWW: http://roberts.ed.psu.edu/users/droberts/drober~1.htm
FRAMES: http://roberts.ed.psu.edu/users/droberts/drframe.htm
===========================================================================
This list is open to everyone. Occasionally, less thoughtful
people send inappropriate messages. Please DO NOT COMPLAIN TO
THE POSTMASTER about these messages because the postmaster has no
way of controlling them, and excessive complaints will result in
termination of the list.
For information about this list, including information about the
problem of inappropriate messages and information about how to
unsubscribe, please see the web page at
http://jse.stat.ncsu.edu/
===========================================================================
