In article <AC09DC4F4DFCD211A83C00805FE6138D3693F0@NHQJPK1EX2>,
Magill, Brett <[EMAIL PROTECTED]> wrote:
>The more general concern about significance testing notwithstanding, I have
>a question about the use of testing, or other inferential statistical
>techniques, in experiments using human subjects, or any other research
>method that does not use probability sampling...
>Now, all of these tests that we run--whether from ANOVA, regression,
>difference of means, correlations, etc.--are based on the assumption that we
>have sampled from a population using some probability sampling procedure.
>And the meaning of p is inextricably linked to the properties of the
>sampling distribution.
The idea that probability is only related to sampling from a
"population" is what needs to be discarded. Instead, one should
take the view that observations are random, which means just that
there are unknown probabilities involved. Deciding which action
to take depends on this consideration. It is only for historical
reasons that the set of observations made is called a sample.
>However, little experimental research with human subjects is done using a
>sample. Most often, in my experience, these studies use volunteer subjects
>or naturally existing groups. These subjects are then randomly assigned to
>treatment and control groups. Yet, in every instance that I know of,
>results are presented with tests of significance. It seems to me that
>outside of the context of probability sampling, these tests have no meaning.
>Despite this, presentation of such results with tests of significance are
>common.
Testing is needed, but not the use of preassigned significance
levels. The fundamental principle of decision making under
uncertainty is that one must
Simultaneously consider all consequences of the
proposed procedure in all states of nature.
This will involve the probabilities of the various actions
being tken in the various states.
This is what you should use to replace "significance" testing.
--
This address is for information only. I do not claim that these views
are those of the Statistics Department or of Purdue University.
Herman Rubin, Dept. of Statistics, Purdue Univ., West Lafayette IN47907-1399
[EMAIL PROTECTED] Phone: (765)494-6054 FAX: (765)494-0558
=================================================================
Instructions for joining and leaving this list and remarks about
the problem of INAPPROPRIATE MESSAGES are available at
http://jse.stat.ncsu.edu/
=================================================================
