I'm pondering a study which purports to show that a specific early intervention works in improving school performance and keeping kids from crime. The design is good, although the results, while positive, are weak.
But there does appear to be some possibility of fiddle in the way the results are analyzed. A group of high-risk children were randomly assigned to intervention and control groups. 15 years after treatment, two objective measures of success were obtained: high school graduation, and criminal record. I'd have gone with a simple independent test of proportions for each measure. When I did, high school graduation was significant, but criminal record was not (p=.09).They didn't do this. They used logistic regression in each case, for graduation controlling both for parental occupation and disruptiveness, and for criminal record, controlling only for parental occupation, The one for graduation was significant, but for criminal record it was p = .06, which they accepted as significant, "although marginal". That's not what's bugging me. What I want to know is if it's justifiable to control for things like parental occupation and disruptiveness in a randomized study. This is ok for correlational research, but why would you want to do it in a randomized study where such factors are already eliminated through randomization? It seems to me they may have done this because it got them close enough to the magic p= .05 to claim it anyway. If that's the only reason, I don't think it's right. Also, once you've controlled in that way, wouldn't that somehow limit the generality of your findings, that they're now restricted to an artificial type of homogeneous population resulting from the controlling? Is there a cost to doing it this way when you don't have to? As our Michael would say, send me something. Stephen ----------------------------------------------------------------- Stephen L. Black, Ph.D. Professor of Psychology, Emeritus Bishop's University e-mail: [EMAIL PROTECTED] 2600 College St. Sherbrooke QC J1M 1Z7 Canada ----------------------------------------------------------------------- ---
