> >>> dennis roberts <[EMAIL PROTECTED]> 04/07 2:46 pm >>>
> i was not suggesting taking away from our arsenal of tricks ... but, since
> i was one of those old guys too ... i am wondering if we were mostly lead
> astray ...?
>
> the more i work with statistical methods, the less i see any meaningful (at
> the level of dominance that we see it) applications of hypothesis
> testing ...
>
> here is a typical problem ... and we teach students this!
>
> 1. we design a new treatment
> 2. we do an experiment
> 3. our null hypothesis is that both 'methods', new and old, produce the
> same results
> 4. we WANT to reject the null (especially if OUR method is better!)
> 5. we DO a two sample t test (our t was 2.98 with 60 df) and reject the
>
> null ... and in our favor!
> 6. what has this told us?
>
> if this is ALL you do ... what it has told you AT BEST is that ... the
> methods probably are not the same ... but, is that the question of
> interest
> to us?
>
> no ... the real question is: how much difference is there in the two
> methods?
>
> our t test does NOT say anything about that
>
> ===========================================================================
The way this world is ---
A master's candidate, or a phD candidate, or a professor,
or a working scientist, has put a lot into his project.
In terms of time, in terms of money, and more important
still, in terms of emotional commitment, (S)he has lived
with this project for two years or more.
That is a source of subjective bias: (S)he WANTS the data to
show something, preferably to support the original idea behind
the research, but even failing that, to show something.
There needs be an objective brake on this wish. An hypothesis
test is that a brake. NOT rejecting the null hypothesis means
that the data has no information (about whatever aspect of the
data the test was designed to look at), STOP THERE; go no
further.
Without some objective brake, the master's student, etc. will
go ahead to say something about the data, even when the test
would have told her(im) there is nothing to say.
Regard rejecting the null hypothesis as permission to look
the data.
SUMMARY:
* Don't be like a certain social sciences graduate
* student at our university who, after failing to reject her
* null hypothesis, nevertheless went on to draw conclusions
* from her data. (Worse than that, her department
* had her seminar presented as a star example.)
bill knight http://www.math.unb.ca/~knight
===========================================================================
This list is open to everyone. Occasionally, less thoughtful
people send inappropriate messages. Please DO NOT COMPLAIN TO
THE POSTMASTER about these messages because the postmaster has no
way of controlling them, and excessive complaints will result in
termination of the list.
For information about this list, including information about the
problem of inappropriate messages and information about how to
unsubscribe, please see the web page at
http://jse.stat.ncsu.edu/
===========================================================================
