Thanks for your replies thus far.
I don't have a specific problem per se. I'm trying to better understand the
GLM and the limitations of ANOVA.
So, in a nutshell, here's my question:
IF a participant is NOT randomly assigned to a level of a factor BECAUSE the
factor in question CANNOT be randomly assigned (i.e. gender, IQ, etc.), WHAT
are the mathematical/statistical consequences.
The domain of interest is Psychology/Behavioral Sciences. The problem is
that many researchers will examine data for group mean differences across
factors that CANNOT be randomly assigned. This seems to violate a basic
assumption of ANOVA- random assignment to treatment groups.
EXAMPLE (purely bogus- I'm making this up on the fly):
A researcher wants to determine whether or not the use of pictures in an
instruction manual has an impact on how well participants can assemble the
device depicted in the instructions. Two instruction manuals are created:
one with pictures and one without (text only). The researcher believes that
the picture instruction manuals will work best for those participants who
score higher on a "mental rotation ability" test. So, participants are given
the mental rotation ability test and are sorted into one of three groups:
low, medium, and high mental rotation ability. The result is a 2x3 BS
design.
Notice that participants were randomly assigned to only one of the two
factors- instruction type. "Assignment" to the other factor was not at all
random; instead, participants were tested on their mental rotation ability
and sorted into one of three groups according to ability.
I would personally use Multiple Regression to tackle this problem (maybe
that's not a reasonable approach?). BUT, I KNOW for a fact that many
behavioral scientists WOULD use ANOVA in this type of situation.
Again, I am NOT working on a project with this type of data; No one I know
is doing the research described above. I would NOT use ANOVA in the above
situation. I HONESTLY want to understand the proper use of ANOVA.
TWO QUESTIONS:
1) Is there a problem with using ANOVA to analyze data when random
assignment was NOT used for one or more of the factors?
2) IF YES, why?
3) IF YES, what are the implications (i.e. Type I error inflation, wrong SS
values, etc.)?
Sorry for the confusion that I have caused. I know enough about stats to be
dangerous.
Thanks
Steve
.
.
=================================================================
Instructions for joining and leaving this list, remarks about the
problem of INAPPROPRIATE MESSAGES, and archives are available at:
. http://jse.stat.ncsu.edu/ .
=================================================================
