On 14 Mar 2003 at 21:23, John Poole wrote: > Not being a statistical mathematician, my question may appear somewhat > naive, but here goes.. > > Does anyone know of a procedure for adjusting the magnitude of a > Bonferroni correction (BC) for the average correlation among the > multiple measures one is performing the statistical tests upon. > Typically, when people use the BC they simply base the correction on > the total number of statistical tests being don. This does not seem to > make sense if the measures are correlated. >
Correction for correlation among the tests can be done, and are being done. I'm none expert on this, but know it is implemented in a package (multcomp) for the R statistical language. Search google for CRAN (the comprehensive R archival network). Kjetil Halvorsen > For example, if two groups are being compared with 20 t-tests on > measures that are uncorrelated with one another, then it makes sense > to adjust alpha for the 20 independent tests being performed. At the > other extreme, if all 20 criterion variables are perfectly correlated > with one another, then the 20 tests will all come out exactly the same > -- equivalent to single t-test, and it would make no sense to increase > alpha as if the conjoint probability of 20 independent events were > being calculated. Most analyses that I see published (including my > own) fall somewhere in between -- 20 t-tests are done on measures that > are known to be moderately correlated with one another. The author may > start with an overall test of significance (such as Hotelling's T from > a MANOVA), and then follow a significant overall effect with the 20 > t-tests. The author then typically does one of the following: (a) > calculates the p-values adjusted for all 20 tests, (b) does not adjust > the t-tests at all, considering the overall test of significance > sufficient, or (c) something in between, such as "alpha= .01 was used, > in view of the large number of tests performed". > > I know there are some interesting alternatives (such as Hochberg's > test that sequentially adjusts alpha as the number of measures are > increased one at a time) -- but this does not really address my > question either: the effect of intercorrelation among measures. I > imagine doing something like a principal components analysis to > identify the "actual" number of underlying orthogonal factors that are > present in ones data and then using that number for the BC. For > example, if 6 principal components account for 95% of the variability > in ones measures, then do a BC as if 6 independent t-test are being > done -- since that is the best estimate of the number of independent > criteria actually present. > > Perhaps a simpler way might be to adjust the BC for the average > correlation among all measures (related to Cronbach's alpha). > > The problem is, I have never seen any indication in the literature of > people trying to deal with this (including lit searches I've done on > the topic). Is my logic flawed, or have I just not looked in the right > places. As one who enjoys statistics, but is not an expert in the > mathematical core of it all, I would be very interested in hearing > people's thoughts on this. > > Thank you much. > > -- > ************************************************* > John H. Poole, Ph.D. > Department of Psychiatry > University of California Medical Center > 4150 Clement Street (116C) > San Francisco, CA 94061, USA > > Phone: 650-281-8851 Fax: 415-750-6996 > Email: [EMAIL PROTECTED]; [EMAIL PROTECTED] > ************************************************* > > > . > . > ================================================================= > Instructions for joining and leaving this list, remarks about the > problem of INAPPROPRIATE MESSAGES, and archives are available at: > . http://jse.stat.ncsu.edu/ . > ================================================================= . . ================================================================= Instructions for joining and leaving this list, remarks about the problem of INAPPROPRIATE MESSAGES, and archives are available at: . http://jse.stat.ncsu.edu/ . =================================================================
