I want to add another piece of theoretical background here - On 19 Dec 2002 09:43:01 -0800, [EMAIL PROTECTED] (Lise DeShea) wrote:
[ ... ] > > First, a fixed-effects ANOVA can have a quite inflated > Type I error rate if the groups are unequal in size. In that case, > you wouldn't want to use Tukey; you'd want the Games-Howell > procedure, which is designed for use with unequal n's. ... [ snip to end of paragraph] > Second, even if you have equal and large n's, severely unequal > variances can lead to increased probability of a Type I error. [ snip, rest] I don't know about "Games-Howell procedure, which is designed for use with unequal n's, ..." -- but the main set of best known, *well-regarded* procedures all share a couple of assumptions: 1) Ns are *equal*: so contrasts make up a balanced set (which is probably symmetrical, and complete in some sense); and 2) Variances are equal: so the shared variance estimate is the reasonable basis for every contrast. To the *extent* that those assumptions are *not* met, you are fudging on the theoretically-legitimate procedure. A reviewer could challenge you on post-hocs, if you those assumptions seemed to lead to bad conclusions, without active justification, in a research report or a proposal. (The fudging lies behind a bit of my distaste for post-hocs; but mainly, the worrying about choices in post-hoc tests, *when there's only 3 or 4 groups,* has seemed to me to be an exercise in splitting hairs, of trying to draw distinctions much finer than should be justified.) Users who like post-hoc tests have felt fairly comfortable in with computer programs that ignore the variances, and which adapt to unequal Ns by assigning the 'harmonic mean' value to every group, for those followup computations. Or else they personally disallow the tests under bad conditions. Again, I don't know the Games-Howell procedure, but G-H can only contribute a systematic set of choices; it cannot justify the choices for the user. For instance, if I notice that one group has N=5, amidst 5 or 10 samples ranging from 200 to 800, my *intelligent choice* will be to eliminate that group, by banishment or folding it into another. In 10 or 15 years, your computer program might suggest that for you - and ask for approval? - but so far that falls under next year's 'data mining' rather than last year's research statistics. -- Rich Ulrich, [EMAIL PROTECTED] http://www.pitt.edu/~wpilib/index.html . . ================================================================= Instructions for joining and leaving this list, remarks about the problem of INAPPROPRIATE MESSAGES, and archives are available at: . http://jse.stat.ncsu.edu/ . =================================================================
