Dear listers,
I reviewed articles using structural equation modeling. To estimate the
rater bias each article was read by another rater. The items were
closely linked to questions used in the relevant literature.
My original plan was to conduct a g-study to estimate the
inter-rater-reliability.
My facets are:
- Rater: R with 2 levels
- Items: I with 21 levels; the items are dichotomous.
- Articles: A with 43 levels.
The design was fully crossed without missing data.
I used the program GT by Pierre Ysewijn.
My problem is that the items cover very different topics and not a
homogenous dimension. Thus I cannot form scales as my items are somewhat
independent of each other.
I would like to generalize over raters and find an answer to the
question:
To what degree are the ratings on the items independent of the rater?
Are the raters interchangeable?
Thus could anybody tell me please what the appropriate term for the
"reliability" estimate is? Which terms have to be in the nominator and
wich terms in the denominator.
Any help is highly appreciated.
Martin Brunner.
===========================================================================
This list is open to everyone. Occasionally, less thoughtful
people send inappropriate messages. Please DO NOT COMPLAIN TO
THE POSTMASTER about these messages because the postmaster has no
way of controlling them, and excessive complaints will result in
termination of the list.
For information about this list, including information about the
problem of inappropriate messages and information about how to
unsubscribe, please see the web page at
http://jse.stat.ncsu.edu/
===========================================================================
