On 14 Aug 2001, Nolan Madson wrote:
> I have a data set of answers to questions on employee performance.
> The answers available are:
>
> Exceeded Expectations
> Met Expectations
> Did Not Meet Expectations
>
> The answers can be assigned weights [that is, scores -- DFB]
> of 3,2,1 (Exceeded, Met, Did Not Meet).
>
> Our client wants to see the results averaged, so, for example, we see
> that all employees in all Ohio offices for the year 2001 have an
> average performance rating of 1.75 while all employees in all Illinois
> offices have an average performance rating of 2.28.
>
> One of my colleagues says that it is not valid to average categorical
> data such as this. His contention is that the only valid form of
> representation is to say that 75% of all respondents ranked Ohio
> employees as having "Met Expectations" or "Exceeded Expectations."
Your colleague is correct about "categorical data". It is not clear
whether he be correct about "data such as this". Your responses are
clearly at least ordinal (in the order you gave them, from most effective
to least effective). The question is whether the differences between
adjacent values are both approximately equal: that is, whether
"Exceeded Expectations" is roughly the same "distance" (in some
conceptual sense) from "Met Expectations" as "Did Not Meet Expectations"
is. (And whether this be the case for all the variables in question.)
These are difficult questions to argue in the abstract, either on
theoretical or empirical grounds -- although for empirical data you
could always carry out a scaling analysis and see if the scale values
thus derived are approximately equidistant.
Probably more important than arguing about whether your data are "only
nominal" (i.e., categorical), or "only ordinal" or of "interval" quality
is, what do your clients (and/or the publics to whom they report)
understand of various styles of reportage? I suspect that some folks
would be much happier with "75% of respondents in Ohio met or exceeded
expectations, while only 60% of respondents in Illinois did so",
together with a statement that the difference is significant (or not),
than with a statement like "all employees in all Ohio offices ... had an
average performance rating of 1.75 while all employees in all Illinois
offices had an average performance rating of 2.28", also with a statement
about the statistical value of the distinction. OTOH, some people prefer
the latter. No good reason not to report in both styles, in fact.
> Can anyone comment on the validity of using averages to report on
> categorical data?
Well, now, as the question is put, the answer is (of course!) that
averages are NOT valid for categorical data (unless the categories are
at least ordinal and more or less equally spaced). But that begs the
question of whether "categorical data" be an adequate description of YOUR
data. I'd judge it is not: it appears to be at least ordinal. The
question whether it be also interval, at least approximately, depends on
the internal representations your respondents made of the questions and
the possible responses, which is a little hard to find out at this point.
However, if (as is often the case) the response medium depicted the three
possible responses on a linear dimension and at equal intervals, it's a
reaosnably good bet that most of your respondents internalized that
dimension accordingly.
> Or point me to reference sources which would help
> clarify the issue? -- Nolan Madson
I doubt that references would help much in dealing with the facts of the
matter, although they might provide you some information and help you to
sound more erudite to your clients... This is essentially a measurement
issue, so appropriate places to look are in textbooks on educational or
psychological measurement.
------------------------------------------------------------------------
Donald F. Burrill [EMAIL PROTECTED]
184 Nashua Road, Bedford, NH 03110 603-471-7128
=================================================================
Instructions for joining and leaving this list and remarks about
the problem of INAPPROPRIATE MESSAGES are available at
http://jse.stat.ncsu.edu/
=================================================================
