Finally a voice of sanity!!!
Henry M. Silvert Ph.D.
Research Statistician
The Conference Board
845 Third Ave.
New York, NY 10022
Phone : (212)339-0438
Fax : (212)836-3825
Email : [EMAIL PROTECTED]
> -----Original Message-----
> From: Alan McLean [SMTP:[EMAIL PROTECTED]]
> Sent: Tuesday, April 11, 2000 7:47 PM
> To: EDSTAT list
> Subject: Hypothesis testing and magic
>
> I have been reading all the back and forth about hypothesis testing with
> some degree of fascination. It's a topic of particular interest to me -
> I presented a paper called 'Hypothesis testing and the Westminster
> System' at the ISI conference in Helsinki last year.
>
> What I find fascinating is the way that hypothesis testing is regarded
> as a technique for finding out 'truth'. Just wave a magic wand, and
> truth will appear out of a set of data (and mutter the magic number 0.05
> while you are waving it....) Hypothesis testing does nothing of the sort
> - of course.
>
> First, hypothesis testing is not restricted to statistics or 'research'.
> If you are told some piece of news or gossip, you automatically check it
> out for plausibility against your knowledge and experience. (This is
> known colloquially as a 'shit filter'.) If you are at a seminar, you
> listen to the presenter in the same way. If what you hear is consistent
> with your knowledge and experience you accept that it is probably true.
> If it is very consistent, you may accept that it IS true. If it is not
> consistent, you will question it, conclude that it is probably not true.
>
> IF the news is something that requires some action on your part, you
> will act according your assessment of the information.
>
> If the news is important to you, and you cannot decide which way to go
> on prior knowledge, you will presumably go and get corroborative
> information, hopefully in some sense objective information.
>
> This describes hypothesis testing almost exactly; the difference is a
> matter of formalism.
>
> Next - a statistical hypothesis test compares two probability models of
> 'reality'. If you are interested in the possible difference between two
> populations on some numeric variable - for example, between heights of
> men and heights of women in some population group - and you choose to
> express the difference in terms of means, you are comparing a model
> which says
> height of a randomly chosen individual = overall mean + random
> fluctuation
> with one which says
> height of a randomly chosen individual = overall mean + factor
> due to sex + random fluctuation
> You then make assumptions about the 'random fluctuations'.
>
> Note that one of these models is embedded within the other - the first
> model is a particular case of the second. It is only in this situation
> that standard hypothesis testing is applicable.
>
> Neither of these models is 'true' - but either or both may be good
> descriptions of the two populations. Good in the sense that if you do
> start to randomly select individuals, the results agree acceptably well
> with what the model predicts. The role of hypothesis testing is to help
> you decide which of these is (PROBABLY) the better model - or if neither
> is.
>
> In standard hypothesis testing, one of these models is 'privileged' in
> that it is assumed 'true' - that is, if neither model is better, then
> you will use the privileged model. In most cases, this means the SIMPLER
> model.
>
> More accurately - if you decide that the models are equally good (or
> bad) you are saying that you cannot distinguish between them on the
> basis of the information and the statistical technique used! To decide
> between them you will need either to use a different technique, or more
> realistically, some other criterion. For example, in a court case, if
> you cannot decide between the models 'Guilty' and 'Innocent', you may
> always choose 'Innocent'.
>
> There is no reason why one model is thus privileged. In my paper I
> stressed my belief that this approach reflects our (and Fisher's)
> cultural heritage rather than any need for it to be that way. One can
> for example express the choice as between the embedded model and the
> embedded model suggested by the data. For a test on the difference
> between two means, this considers the models mu(diff) = 0 and mu(diff) =
> xbar. The interesting thing is that this is what we actually do!
> although it is dressed up in the language and technique of the general
> model mu(diff) not= 0. (This dressing up is a lot of the reason why
> students have trouble with hypothesis testing.)
>
> To conclude: hypothesis testing is NECESSARY. We do it all the time.
> Assessment of effect sizes is also necessary, but the two should not be
> confused.
>
> Regards,
> Alan
>
> --
> Alan McLean ([EMAIL PROTECTED])
> Department of Econometrics and Business Statistics
> Monash University, Caulfield Campus, Melbourne
> Tel: +61 03 9903 2102 Fax: +61 03 9903 2007
>
>
>
>
> ==========================================================================
> =
> This list is open to everyone. Occasionally, less thoughtful
> people send inappropriate messages. Please DO NOT COMPLAIN TO
> THE POSTMASTER about these messages because the postmaster has no
> way of controlling them, and excessive complaints will result in
> termination of the list.
>
> For information about this list, including information about the
> problem of inappropriate messages and information about how to
> unsubscribe, please see the web page at
> http://jse.stat.ncsu.edu/
> ==========================================================================
> =
===========================================================================
This list is open to everyone. Occasionally, less thoughtful
people send inappropriate messages. Please DO NOT COMPLAIN TO
THE POSTMASTER about these messages because the postmaster has no
way of controlling them, and excessive complaints will result in
termination of the list.
For information about this list, including information about the
problem of inappropriate messages and information about how to
unsubscribe, please see the web page at
http://jse.stat.ncsu.edu/
===========================================================================
