Robert McGrath ([EMAIL PROTECTED]) wrote:
:
: If what you mean is that really large samples can lead to distorted
: results of significance tests, I would disagree. The problem is not
: that the sample is too big, but that Significance tests are interpreted in
: inappropritae ways when readers assume statistical significance equals
: clinical significance.
I've come to believe that terms like "clinical significance" and
"practical significance" unintentionally appeal to psychological set and
should be phrased as "clinical *importance*" or "practical *importance*"
instead. It takes a fair amount of "statistical maturity" to avoid
conflating the two uses of "significance."
The problem you describe is actually one of letting one's research
hypotheses be dictated by the tools one is familiar with ("if all you
have is a hammer, you put the cart before the horse"). "Is the effect of
framagublification nonzero" is simply not an interesting question. "Is
the effect of framagublification large enough to be meaningful according
to our current understanding of Turner-Gates pathology" *is* an
interesting question, and the fact that the former can be answered by
merely clicking on menu items while the latter can't is about as relevant
to the research as the fact that student's heights can be easily and
repeatably measured is to the instructor's grading system. "This
question is too hard to answer, so I'll search for the answer to a
different question" is the motto of the coward.
One practical problem with really large samples nobody's mentioned yet;
lots more missing data, particularly data missing *not* at random.
Non-response, procedural glitches that wipe out big chunks of data,
compromises when you realize you're about to exceed your budget, etc.
The bigger the project, the greater the Murphic exposure.
===========================================================================
This list is open to everyone. Occasionally, less thoughtful
people send inappropriate messages. Please DO NOT COMPLAIN TO
THE POSTMASTER about these messages because the postmaster has no
way of controlling them, and excessive complaints will result in
termination of the list.
For information about this list, including information about the
problem of inappropriate messages and information about how to
unsubscribe, please see the web page at
http://jse.stat.ncsu.edu/
===========================================================================
