>From my understanding, there are three popular ways to analyze the following design
>(let's call it the pretest-posttest control-group design):
R Pretest Treatment Posttest
R Pretest Control Posttest
In the social sciences (e.g., see Pedhazur's popular regression text), the most
popular analysis seems to be to run a GLM (this version is often called an ANCOVA),
where Y is the posttest measure, X1 is the pretest measure, and X2 is the treatment
variable. Assuming that X1 and X2 do not interact, ones' estimate of the treatment
effect is given by B2 (i.e., the partial regression coefficient for the treatment
variable which controls for adjusts for pretest differences).
Another traditionally popular analysis for the design given above is to compute a new,
gain score variable (posttest minus pretest) for all cases and then run a GLM (ANOVA)
to see if the difference between the gains (which is the estimate of the treatment
effect) is statistically significant.
The third, and somewhat less popular (?) way to analyze the above design is to do a
mixed ANOVA model (which is also a GLM but it is harder to write out), where Y is the
posttest, X1 is "time" which is a repeated measures variable (e.g., time is 1 for
pretest and 2 for posttest for all cases), and X2 is the between group, treatment
variable. In this case one looks for treatment impact by testing the statistical
significance of the two-way interaction between the time and the treatment variables.
Usually, you ask if the difference between the means at time two is greater than the
difference at time one (i.e., you hope that the treatment lines will not be parallel)
Results will vary depending on which of these three approaches you use, because each
approach estimates the counterfactual in a slightly different way. I believe it was
Reichardt and Mark (in Handbook of Applied Social Research Methods) that suggested
analyzing your data using more than one of these three statistical methods.
I'd be interested in any thoughs you have about these three approaches.
Take care,
Burke Johnson
http://www.coe.usouthal.edu/bset/Faculty/BJohnson/Burke.html
=================================================================
Instructions for joining and leaving this list and remarks about
the problem of INAPPROPRIATE MESSAGES are available at
http://jse.stat.ncsu.edu/
=================================================================
