Jerry Dallal <[EMAIL PROTECTED]> wrote: >> > ... I make the point on my web page that P values make sense >> >only in the context of fixed level testing. Otherwise, you leave >> >yourself open to Harold Jeffrey's criticism that those who do >> >hypothesis testing act on the basis of rare events that they don't >> >see. If one says only that the P value is the probability of seeing >> >results as or more extreme than what was observed, one leaves one's >> >self open to the question, "Why should I care about the probability >> >results I haven't seen?" >>
Radford Neal wrote: >> I think you can't avoid this criticism so easily. The levels of the >> tests that you are using to determine the p value are determined by >> what these tests would do if the data had been other than what you saw, >> and on the probabilities of these data sets that weren't actually seen. Jerry Dallal <[EMAIL PROTECTED]> wrote: >I think you can. All you have to say is, "I believe in fixed level >testing." People can argue over the validity of fixed level >testing, but the acceptance of fixed level testing (and the >interpretation of P values in that context) avoids Jeffreys' >criticism. If you refuse to believe in fixed level testing, then >you have a problem that's bigger than just P values. Why would not believing in fixed level testing be a problem? There is even less justification for fixed level testing than there is for use of p-values in general. And in fact, NOBODY believes in fixed level testing. There isn't a scientist anywhere who cares only whether p<0.05 (or whatever level), and is entirely indifferent to whether p=0.049 or p=0.001. (Well, maybe there is a scientist somewhere who thinks that way, but only because they're REALLY stupid.) The argument is pure sophistry in any case, however. Jeffrey's criticism obviously applies just as much to people doing fixed level testing. Saying that you are going to accept fixed level testing as axiomatic doesn't avoid the criticism, it's just a declaration that you're going to ignore it. But if you want to ignore Jeffrey's criticism, you can just declare that you're ignoring it without having to commit to fixed level testing. Radford Neal . . ================================================================= Instructions for joining and leaving this list, remarks about the problem of INAPPROPRIATE MESSAGES, and archives are available at: . http://jse.stat.ncsu.edu/ . =================================================================
