On Sat, 22 Jul 2000, Mike Hewitt wrote (edited):
> I am looking for assistance in interpreting results of a study. ...
> I performed a GLM-repeated measures with three factors (modeling,
> self-listening, self-evaluation) in addition to a repeated measure
> (test). There was a significant interaction for test x modeling x
> self-evaluation. There was also a significant result for test x
> modeling. Does the higher-order interaction negate the results the
> "main effects" or lower-order interaction?
"Negate" is too strong a term, in general; although it is possible, and
you do not supply enough information for the reader to tell. What is
true in general is:
+ A 2-way interaction (AxB, say) implies that the effect of A is
different at different levels of B. The difference(s) may be
only of magnitude (the effect is always in the same direction,
but is stronger or weaker depending on B; and possibly there
is NO effect of A at one or more levels of B), or also of
direction (e.g., A1 > A2 at B1, but A1 < A2 at B2; etc.).
+ A 3-way interaction (AxBxC) implies that the 2-way AB interaction
differs at different levels of C. (Again, the difference(s) may
be only of magnitude (the same shape of interaction, only
sometimes stronger, sometimes weaker), or of existence (there's an
AB interaction at C1, but none at C2), or of direction.)
Equivalently, the 2-way AC interaction differs at different
levels of B; and the 2-way BC interaction differs at different
levels of A. Notwithstanding the equivalence, one of these ways
of looking at the situation may be more useful or convenient than
one or both of the other ways.
That having been said, it is nonetheless possible (as someone else has
pointed out) for a main effect to be so much larger than the interactions
as to overwhelm them, so that the interactions are as it were mere
appoggiaturas on the cantus firmus supplied by the main effect; equally,
it is possible for an interaction to overwhelm one or more main effects,
as when the subject of a fugue is inverted at a subsequent entry.
I have never found it possible to make reasonable sense out of 3-way (or
higher) ANOVAs in the presence of significant interaction without
plotting the subgroup means corresponding to the highest-order
interaction that seems to be modifying the effects of interest; and
usually I've ended up plotting in all possible directions (e.g., plot the
AB interaction separately for each level of C; plot the AC interaction
separately for each level of B; plot the BC interaction separately for
each level of A).
Remember, in all of this, that the decomposition of the total SS around
the means of the smallest cells into the standard "main effects", "2-way
interactions", "3-way interactions", etc., is quite arbitrary. It may
even be useful, especially in balanced designs where these several
effects are all mutually orthogonal. But there are other decompositions
(and even other orthogonal decompositions) that may be easier to describe
and/or interpret. One should try to be sensitive to what the universe is
trying to tell one, even (or, perhaps, especially?) if it doesn't fit the
standard models.
> Specifically, musicians who listened to "model" performance improved
> their performance more than those that did not listen to a model.
I take it then that the "modeling" factor had 2 levels:
presence/absence.
> Great. For the interaction (test x modeling x self-evaluation), the
> modeling/self-evaluation group improved more than did the no
> modeling/self-evaluation group (reinforcing the results for modeling
> only). HOWEVER, the same result did not occur for the groups that did
> not self-evaluate. They improved similarly to each other.
And this sounds as though self-evaluation also had 2 levels:
presence/absence. Would I be correct in guessing that "test" also has 2
levels, like maybe pretest/posttest? (Since you speak of "improved"
performance.) Then you might find it easier to interpret if instead of
conducting a 3-way ANOVA with repeated measures on "test", using posttest
score as the response variable, you conducted a 2-way ANOVA on the change
(posttest minus pretest).
> So...listening to a model is more effective than not listening to one
> when there is no-self-evaluation.
Sorry! This is NOT what you reported above. If I have correctly
understood your statement: "the modeling/self-evaluation group improved
more than did the no modeling/self-evaluation group (reinforcing the
results for modeling only)", listening to a model is more effective than
not listening to one WHEN THERE IS SELF-EVALUATION. (I don't understand
your parenthetical reference to "results for modeling only" -- I suspect
you mean the main effect for modeling, but that's not "for modeling
only", it's for an average between those who self-evaluate and those who
don't.)
Your subsequent statement, "... for the groups that did not
self-evaluate. They improved similarly to each other" seems to say that
for the groups that did not self-evaluate, there was no effect of
modelling: the improvement was "similar" (= "not significantly
different"?) between modelling and not modelling in the absence of
self-evaluation.
> What then does this mean as far as the results for the test x modeling
> result?
For that, more information is needed. Is there any evidence of a "floor"
effect at pretest, or a "ceiling" effect at posttest? If so, this may
entirely explain the pattern of results, as another correspondent has
hinted. (It also may not; depends on what that pattern is!)
In fact, what WAS "the test x modeling result" ?
> I guess my question is does a higher-level interaction
> "overrule" a lower-level interaction?
And, as you must by now have gathered, the answer is a firm "Maybe".
'Twould be easier to make sense of your question, and offer possibly
useful answers, if you'd care to supply the eight (?) cell means in
question, and include the minimum and maximum possible scores.
------------------------------------------------------------------------
Donald F. Burrill [EMAIL PROTECTED]
348 Hyde Hall, Plymouth State College, [EMAIL PROTECTED]
MSC #29, Plymouth, NH 03264 603-535-2597
184 Nashua Road, Bedford, NH 03110 603-471-7128
=================================================================
Instructions for joining and leaving this list and remarks about
the problem of INAPPROPRIATE MESSAGES are available at
http://jse.stat.ncsu.edu/
=================================================================
