Re: Presenting results of categorical data?
On 14 Aug 2001, Nolan Madson wrote: > I have a data set of answers to questions on employee performance. > The answers available are: > > Exceeded Expectations > Met Expectations > Did Not Meet Expectations > > The answers can be assigned weights [that is, scores -- DFB] > of 3,2,1 (Exceeded, Met, Did Not Meet). > > Our client wants to see the results averaged, so, for example, we see > that all employees in all Ohio offices for the year 2001 have an > average performance rating of 1.75 while all employees in all Illinois > offices have an average performance rating of 2.28. > > One of my colleagues says that it is not valid to average categorical > data such as this. His contention is that the only valid form of > representation is to say that 75% of all respondents ranked Ohio > employees as having "Met Expectations" or "Exceeded Expectations." Your colleague is correct about "categorical data". It is not clear whether he be correct about "data such as this". Your responses are clearly at least ordinal (in the order you gave them, from most effective to least effective). The question is whether the differences between adjacent values are both approximately equal: that is, whether "Exceeded Expectations" is roughly the same "distance" (in some conceptual sense) from "Met Expectations" as "Did Not Meet Expectations" is. (And whether this be the case for all the variables in question.) These are difficult questions to argue in the abstract, either on theoretical or empirical grounds -- although for empirical data you could always carry out a scaling analysis and see if the scale values thus derived are approximately equidistant. Probably more important than arguing about whether your data are "only nominal" (i.e., categorical), or "only ordinal" or of "interval" quality is, what do your clients (and/or the publics to whom they report) understand of various styles of reportage? I suspect that some folks would be much happier with "75% of respondents in Ohio met or exceeded expectations, while only 60% of respondents in Illinois did so", together with a statement that the difference is significant (or not), than with a statement like "all employees in all Ohio offices ... had an average performance rating of 1.75 while all employees in all Illinois offices had an average performance rating of 2.28", also with a statement about the statistical value of the distinction. OTOH, some people prefer the latter. No good reason not to report in both styles, in fact. > Can anyone comment on the validity of using averages to report on > categorical data? Well, now, as the question is put, the answer is (of course!) that averages are NOT valid for categorical data (unless the categories are at least ordinal and more or less equally spaced). But that begs the question of whether "categorical data" be an adequate description of YOUR data. I'd judge it is not: it appears to be at least ordinal. The question whether it be also interval, at least approximately, depends on the internal representations your respondents made of the questions and the possible responses, which is a little hard to find out at this point. However, if (as is often the case) the response medium depicted the three possible responses on a linear dimension and at equal intervals, it's a reaosnably good bet that most of your respondents internalized that dimension accordingly. > Or point me to reference sources which would help > clarify the issue?-- Nolan Madson I doubt that references would help much in dealing with the facts of the matter, although they might provide you some information and help you to sound more erudite to your clients... This is essentially a measurement issue, so appropriate places to look are in textbooks on educational or psychological measurement. Donald F. Burrill [EMAIL PROTECTED] 184 Nashua Road, Bedford, NH 03110 603-471-7128 = Instructions for joining and leaving this list and remarks about the problem of INAPPROPRIATE MESSAGES are available at http://jse.stat.ncsu.edu/ =
Re: Presenting results of categorical data?
Donald Burrill wrote: I agree on all of this. I'd add that at issue is whether people find the mean format useful, whether it is misleading. I'd use -1, 0 and +1, rather than 1-3. In this case the mean gives you at-a-glance summary of the extent to which the people who exceeded expectations outnumbered those who failed to reach them. It shouldn't be too misleading unless people are using the scale in a very odd way. (OTOH a graphical method is probably even better). Thom = Instructions for joining and leaving this list and remarks about the problem of INAPPROPRIATE MESSAGES are available at http://jse.stat.ncsu.edu/ =
RE: Presenting results of categorical data?
I would like to add that with this kind of data we use the median instead of the average. Henry M. Silvert Ph.D. Research Statistician The Conference Board 845 3rd. Avenue New York, NY 10022 Tel. No.: (212) 339-0438 Fax No.: (212) 836-3825 > -Original Message- > From: Donald Burrill [SMTP:[EMAIL PROTECTED]] > Sent: Wednesday, August 15, 2001 3:34 AM > To: Nolan Madson > Cc: [EMAIL PROTECTED] > Subject: Re: Presenting results of categorical data? > > On 14 Aug 2001, Nolan Madson wrote: > > > I have a data set of answers to questions on employee performance. > > The answers available are: > > > > Exceeded Expectations > > Met Expectations > > Did Not Meet Expectations > > > > The answers can be assigned weights [that is, scores -- DFB] > > of 3,2,1 (Exceeded, Met, Did Not Meet). > > > > Our client wants to see the results averaged, so, for example, we see > > that all employees in all Ohio offices for the year 2001 have an > > average performance rating of 1.75 while all employees in all Illinois > > offices have an average performance rating of 2.28. > > > > One of my colleagues says that it is not valid to average categorical > > data such as this. His contention is that the only valid form of > > representation is to say that 75% of all respondents ranked Ohio > > employees as having "Met Expectations" or "Exceeded Expectations." > > Your colleague is correct about "categorical data". It is not clear > whether he be correct about "data such as this". Your responses are > clearly at least ordinal (in the order you gave them, from most effective > to least effective). The question is whether the differences between > adjacent values are both approximately equal: that is, whether > "Exceeded Expectations" is roughly the same "distance" (in some > conceptual sense) from "Met Expectations" as "Did Not Meet Expectations" > is. (And whether this be the case for all the variables in question.) > These are difficult questions to argue in the abstract, either on > theoretical or empirical grounds -- although for empirical data you > could always carry out a scaling analysis and see if the scale values > thus derived are approximately equidistant. > > Probably more important than arguing about whether your data are "only > nominal" (i.e., categorical), or "only ordinal" or of "interval" quality > is, what do your clients (and/or the publics to whom they report) > understand of various styles of reportage? I suspect that some folks > would be much happier with "75% of respondents in Ohio met or exceeded > expectations, while only 60% of respondents in Illinois did so", > together with a statement that the difference is significant (or not), > than with a statement like "all employees in all Ohio offices ... had an > average performance rating of 1.75 while all employees in all Illinois > offices had an average performance rating of 2.28", also with a statement > about the statistical value of the distinction. OTOH, some people prefer > the latter. No good reason not to report in both styles, in fact. > > > Can anyone comment on the validity of using averages to report on > > categorical data? > > Well, now, as the question is put, the answer is (of course!) that > averages are NOT valid for categorical data (unless the categories are > at least ordinal and more or less equally spaced). But that begs the > question of whether "categorical data" be an adequate description of YOUR > data. I'd judge it is not: it appears to be at least ordinal. The > question whether it be also interval, at least approximately, depends on > the internal representations your respondents made of the questions and > the possible responses, which is a little hard to find out at this point. > However, if (as is often the case) the response medium depicted the three > possible responses on a linear dimension and at equal intervals, it's a > reaosnably good bet that most of your respondents internalized that > dimension accordingly. > > > Or point me to reference sources which would help > > clarify the issue? -- Nolan Madson > > I doubt that references would help much in dealing with the facts of the > matter, although they might provide you some information and help you to > sound more erudite to your clients... This is essentially a measurement > issue, so appropriate places to look are in textbooks on educational or > psychological measurement. > > > Donald F. Burrill [EMAIL PROTECTED] > 184 Nashua Road, Bedford, NH 03110 603-471-7128 > > > > = > Instructions for joining and leaving this list and remarks about > the problem of INAPPROPRIATE MESSAGES are available at > http://jse.stat.ncsu.edu/ > ==
RE: Presenting results of categorical data?
On Wed, 15 Aug 2001, Silvert, Henry wrote:
> I would like to add that with this kind of data
[three ordered categories per item, scored {3,2,1}]
> we use the median instead of the average.
I cannot resist pointing out two things:
(1) At the level of individual items, the median of three ordered
categories is pretty imprecise UNLESS one makes some interesting
assumptions about the scale values that are tantamount to treating them
as interval data;
(2) If the items are summed into a total score, they are being treated
as interval (and the total is a fortiori interval).
In either case, it is not clear whether the additional effort involved
in calculating medians (and their standard errors?), rather than means
etc., is worth one's while...
Donald F. Burrill [EMAIL PROTECTED]
184 Nashua Road, Bedford, NH 03110 603-471-7128
=
Instructions for joining and leaving this list and remarks about
the problem of INAPPROPRIATE MESSAGES are available at
http://jse.stat.ncsu.edu/
=
Re: Presenting results of categorical data?
"Silvert, Henry" wrote: > > I would like to add that with this kind of data [three-level ordinal] > we use the median instead of the average. Might I suggest that *neither* is appropriate for most purposes? In many ways, three-level ordinal data is like dichotomous data - though there are a couple critical differences. Nobody would use the median (which essentially coincides with the mode) for dichotomous data unless thay had a very specific reason for wanting that specific bit of information (and I use the word "bit" in its technical sense.) By contrast, the mean (=proportion) is a lossless summary of the data up to permutation (and hence a sufficient statistic for any inference that assumes an IID model) - about as good as you can get. With three levels, the mean is of course hopelessly uninterpretable without some way to establish the relative distances between the levels. However, the median is still almost information-free (total calorie content per 100-gram serving <= log_2(3) < 2 bits). I would suggest that unless there is an extremely good reason to summarize the data as ONE number, three-level ordinal data should be presented as a frequency table. Technically one row could be omitted but there is no strong reason to do so. "What about inference?" Well, one could create various nice modifications on a confidence interval; most informative might be a confidence (or likelihood) region within a homogeneous triangle plot, but a double confidence interval for the two cutoff points would be easier. As for testing - first decide what your question is. If it *is* really "are the employees in state X better than those in state Y?" you must then decide what you mean by "better". *Do* you give any weight to the number of "exceeded expectations" responses? Do you find 30-40-30 to be better than 20-60-20, equal, or worse? What about 20-50-30? If you can answer all questions of this type, by the way, you may be ready to establish a scale to convert your data to ratio. If you can't, you will have to forego your hopes of One Big Hypothesis Test. I do realize that we have a cultural belief in total ordering and single parameters, and we tend to take things like stock-market and cost-of-living indices, championships and MVP awards, and quality- of- living indices, more seriously than we should. We tend to prefer events not to end in draws; sports that can end in a draw tend to have (sometimes rather silly) tiebreaking mechanisms added to them. Even in sports (chess, boxing) in which the outcomes of (one-on-one) events are known to be sometimes intransitive, we insist on "finding a champion". But perhaps the statistical community ought to take the lead in opposing this bad habit! To say that "75% of all respondents ranked Ohio employees as having 'Met Expectations' or 'Exceeded Expectations.' ", as a single measure, is not a great deal better than taking the mean in terms of information content *or* arbitrariness. Pooling two levels and taking the proportion is just taking the mean with a 0-1-1 coding. It says, in effect, that we will consider (Exceed - Meet)/(Meet - Fail) = 0 while taking the mean with a 0-1-2 coding says that we will consider (Exceed - Meet)/(Meet - Fail) = 1. One is no less arbitrary than the other. (An amusing analogy can be drawn with regression, when users of OLS regression, implicitly assuming all the variation to be in the dependent variable, sometimes criticise the users of neutral regression for being "arbitrary" in assuming the variance to be equally divided.) -Robert Dawson = Instructions for joining and leaving this list and remarks about the problem of INAPPROPRIATE MESSAGES are available at http://jse.stat.ncsu.edu/ =
Re: Presenting results of categorical data?
I do not see how (probabilistic) inference is appropriate here at all. I assume that _all_ employees are rated. There is no sampling, random or otherwise. Jon Cryer At 11:14 AM 8/15/01 -0300, you wrote: > > >"Silvert, Henry" wrote: >> >> I would like to add that with this kind of data [three-level ordinal] >> we use the median instead of the average. > > Might I suggest that *neither* is appropriate for most purposes? In >many ways, three-level ordinal data is like dichotomous data - though >there are a couple critical differences. > > Nobody would use the median (which essentially coincides with the >mode) for dichotomous data unless thay had a very specific reason for >wanting that specific bit of information (and I use the word "bit" in >its technical sense.) By contrast, the mean (=proportion) is a lossless >summary of the data up to permutation (and hence a sufficient statistic >for any inference that assumes an IID model) - about as good as you can >get. > > With three levels, the mean is of course hopelessly uninterpretable >without some way to establish the relative distances between the levels. >However, the median is still almost information-free (total calorie >content per 100-gram serving <= log_2(3) < 2 bits). I would suggest >that unless there is an extremely good reason to summarize the data as >ONE number, three-level ordinal data should be presented as a frequency >table. Technically one row could be omitted but there is no strong >reason to do so. > > "What about inference?" Well, one could create various nice >modifications on a confidence interval; most informative might be a >confidence (or likelihood) region within a homogeneous triangle plot, >but a double confidence interval for the two cutoff points would be >easier. As for testing - first decide what your question is. If it *is* >really "are the employees in state X better than those in state Y?" you >must then decide what you mean by "better". *Do* you give any weight to >the number of "exceeded expectations" responses? Do you find 30-40-30 >to be better than 20-60-20, equal, or worse? What about 20-50-30? If >you can answer all questions of this type, by the way, you may be ready >to establish a scale to convert your data to ratio. If you can't, you >will have to forego your hopes of One Big Hypothesis Test. > > I do realize that we have a cultural belief in total ordering and >single parameters, and we tend to take things like stock-market and >cost-of-living indices, championships and MVP awards, and quality- of- >living indices, more seriously than we should. We tend to prefer events >not to end in draws; sports that can end in a draw tend to have >(sometimes rather silly) tiebreaking mechanisms added to them. Even in >sports (chess, boxing) in which the outcomes of (one-on-one) events are >known to be sometimes intransitive, we insist on "finding a champion". >But perhaps the statistical community ought to take the lead in opposing >this bad habit! > > To say that "75% of all respondents ranked Ohio employees as having >'Met Expectations' or 'Exceeded Expectations.' ", as a single measure, >is not a great deal better than taking the mean in terms of information >content *or* arbitrariness. Pooling two levels and taking the >proportion is just taking the mean with a 0-1-1 coding. It says, in >effect, that we will consider > > (Exceed - Meet)/(Meet - Fail) = 0 > >while taking the mean with a 0-1-2 coding says that we will consider > > (Exceed - Meet)/(Meet - Fail) = 1. > >One is no less arbitrary than the other. (An amusing analogy can be >drawn with regression, when users of OLS regression, implicitly assuming >all the variation to be in the dependent variable, sometimes criticise >the users of neutral regression for being "arbitrary" in assuming the >variance to be equally divided.) > > -Robert Dawson > > >= >Instructions for joining and leaving this list and remarks about >the problem of INAPPROPRIATE MESSAGES are available at > http://jse.stat.ncsu.edu/ >= > ___ --- | \ Jon Cryer, Professor Emeritus ( ) Dept. of Statistics www.stat.uiowa.edu/~jcryer \\_University and Actuarial Science office 319-335-0819 \ * \of Iowa The University of Iowa home 319-351-4639 \/Hawkeyes Iowa City, IA 52242 FAX319-335-3017 |__ ) --- V "It ain't so much the things we don't know that get us into trouble. It's the things we do know that just ain't so." --Artemus Ward = Instructions for joining and leaving t
Re: Presenting results of categorical data?
Jon Cryer wrote:
>
> I do not see how (probabilistic) inference is appropriate here at all.
Oh, it never is (strictly), outside of a few industrial
applications. Nobody ever took a random equal-probability sample from
all turnips, all cancer patients, all batches of stainless steel, all
white mice, or all squirrels. However, there are good common-sense
reasons to believe that *some* convenience samples will act enough like
true random samples to be useful. Otherwise we could stop teaching
statistics to biology and psychology students.
Indeed, it could be argued that pure science *only* makes inferences
about populations for which no sampling frame can be constructed. Yes,
you may be as random as you like about choosing a control group of 50
from your 100 white mice. But unless you title your paper
"The effect of resublimated thiotimoline on the reaction times of one
particular population of 100 white mice (now deceased)"
you are making inferences about a *different* population - that of all
white mice - and the only reason to extend the inference to that
population (rather to that of termites, or light bulbs) is because your
original 100 are a convenience sample from the population of all white
mice.
> I assume that _all_ employees are rated. There is no sampling, random
> or otherwise.
True. However, it seems reasonable to consider such a data set as a
pseudo-random sample from the "Platonic" set of "all potential employees
under this system" and interpret the inference in terms of the existence
of a pattern.
For instance: if I determine the mean height of all named peaks in the
Rockies whose names start with A,B,...,M, and compare that with the mean
height of all named peaks whose names start with N,...,Z, presumably one
mean will be greater. However, this is presumably a "fact without a
reason"; and many of us would place moderate wagers, if the odds were
right, on the difference between the means behaving much like the
difference between two random samples from the same population.
Moreover, should it turn out that this was *not* the case, no doubt you
(like most of us) would ask "why?" (perhaps the most prominent peaks had
been named for saints ("S")? Perhaps a couple super-high peaks
dominated and invalidated distributional assumptions?)
Now, if the same comparison were done between the Rockies and the
Himalayas, we would *not* be surprised to see a difference bigger than
the t distribution might predict. (Again - these are not random
samples.) And why?
"Because the Himalayas are higher than the Rockies." (Duh!)
Ah, but (let's say) the A-M Rockies are bigger than the N-Z Rockies.
"Ah, but that's just chance, there's no real pattern there."
Precisely.
To summarize: In most disciplines, there are no true random samples.
There are better and worse convenience samples
In most disciplines, there are no sampling frames. There are
populations, usually to some extent abstract or at least unlistable.
Inference is ultimately not about populations, but about patterns. A
property of an entire well-defined population is one example of a
pattern; there are others.
It behooves us to accept this and work with it, rather than to delude
ourselves that we sometimes have a true random sample.
-Robert Dawson
=
Instructions for joining and leaving this list and remarks about
the problem of INAPPROPRIATE MESSAGES are available at
http://jse.stat.ncsu.edu/
=
Categorical data Take 2
The discussion of categorical data has got me thinking about a project I am about begin. The goal is to use a variety of individual predictors (IQ, previous work experience, education, personality) to develop a model to predict "success" after a vocational rehabilitation program for psychiatric patients. The problem is how to define success. The current data provides 5 possible outcomes: Full-time employment, part-time employment, ongoing education, volunteer work, or no change. Clearly there is no argument to be made that these are linear, but even ordinal is questionable. I had thought of using a number of logit regression analyses for the various outcomes. Or, to use linear regression, rescaling as number of hours per week employed, and combining the two employment outcomes; scaling education training in terms of program length; combining volunteer and no change into number of hours involved in productive activity. That would give be three outcomes. Any suggestions would be much appreciated! Melady Preece, Ph.D.
RE: Categorical data Take 2
Why not a discriminant analysis? You might want to develop profiles of people who go into the 5 different success categories -- although they might all be equally sucess except for the last one. Henry M. Silvert Ph.D. Research Statistician The Conference Board 845 3rd. Avenue New York, NY 10022 Tel. No.: (212) 339-0438 Fax No.: (212) 836-3825 > -Original Message- > From: Melady Preece [SMTP:[EMAIL PROTECTED]] > Sent: Wednesday, August 15, 2001 10:57 AM > To: [EMAIL PROTECTED] > Subject: Categorical data Take 2 > > The discussion of categorical data has got me thinking about a project I > am about begin. The goal is to use a variety of individual predictors > (IQ, previous work experience, education, personality) to develop a model > to predict "success" after a vocational rehabilitation program for > psychiatric patients. > > The problem is how to define success. The current data provides 5 > possible outcomes: Full-time employment, part-time employment, ongoing > education, volunteer work, or no change. Clearly there is no argument to > be made that these are linear, but even ordinal is questionable. > > I had thought of using a number of logit regression analyses for the > various outcomes. Or, to use linear regression, rescaling as number of > hours per week employed, and combining the two employment outcomes; > scaling education training in terms of program length; combining volunteer > and no change into number of hours involved in productive activity. That > would give be three outcomes. > > Any suggestions would be much appreciated! > > Melady Preece, Ph.D. > = Instructions for joining and leaving this list and remarks about the problem of INAPPROPRIATE MESSAGES are available at http://jse.stat.ncsu.edu/ =
RE: Categorical data Take 2
If your going to use discriminant analysis you will need a lot of data and it does assume the predictors are multivariate normal. Generalized linear models would seem best, particularly in the event that you don't know if they are ordinal. You can do a multinomial followed by a cummulative logit model to see if the data are approximately ordinal. Paul R. Swank, Ph.D. Professor Developmental Pediatrics UT Houston Health Science Center -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]On Behalf Of Silvert, Henry Sent: Wednesday, August 15, 2001 10:36 AM To: 'Melady Preece'; [EMAIL PROTECTED] Subject: RE: Categorical data Take 2 Why not a discriminant analysis? You might want to develop profiles of people who go into the 5 different success categories -- although they might all be equally sucess except for the last one. Henry M. Silvert Ph.D. Research Statistician The Conference Board 845 3rd. Avenue New York, NY 10022 Tel. No.: (212) 339-0438 Fax No.: (212) 836-3825 > -Original Message- > From: Melady Preece [SMTP:[EMAIL PROTECTED]] > Sent: Wednesday, August 15, 2001 10:57 AM > To: [EMAIL PROTECTED] > Subject: Categorical data Take 2 > > The discussion of categorical data has got me thinking about a project I > am about begin. The goal is to use a variety of individual predictors > (IQ, previous work experience, education, personality) to develop a model > to predict "success" after a vocational rehabilitation program for > psychiatric patients. > > The problem is how to define success. The current data provides 5 > possible outcomes: Full-time employment, part-time employment, ongoing > education, volunteer work, or no change. Clearly there is no argument to > be made that these are linear, but even ordinal is questionable. > > I had thought of using a number of logit regression analyses for the > various outcomes. Or, to use linear regression, rescaling as number of > hours per week employed, and combining the two employment outcomes; > scaling education training in terms of program length; combining volunteer > and no change into number of hours involved in productive activity. That > would give be three outcomes. > > Any suggestions would be much appreciated! > > Melady Preece, Ph.D. > = Instructions for joining and leaving this list and remarks about the problem of INAPPROPRIATE MESSAGES are available at http://jse.stat.ncsu.edu/ = = Instructions for joining and leaving this list and remarks about the problem of INAPPROPRIATE MESSAGES are available at http://jse.stat.ncsu.edu/ =
Re: Presenting results of categorical data?
On 14 Aug 2001 15:45:04 -0700, [EMAIL PROTECTED] (Nolan Madson) wrote: > I have a data set of answers to questions on employee performance. > The answers available are: > > Exceeded Expectations > Met Expectations > Did Not Meet Expectations > > The answers can be assigned weights of 3,2,1 (Exceeded, Met, Did Not > Meet). > > Our client wants to see the results averaged, so, for example, we see > that all employees in all Ohio offices for the year 2001 have an > average performance rating of 1.75 while all employees in all Illinois > offices have an average performance rating of 2.28. > > One of my colleagues says that it is not valid to average categorical > data such as this. His contention is that the only valid form of > representation is to say that 75% of all respondents ranked Ohio > employees as having "Met Expectations" or "Exceeded Expectations." Here is a perspective not emphasized so far, in the half dozen responses I have read. At the start, information and sampling is rooted in *respondents* and not employees. Is there one value used, as the average for each respondent, which is then averaged? Unless each respondent rated the same number of employees, you have two obvious choices for the weighting. But a) one of those doesn't give you "independence of ratings"; b) the other one doesn't give you scores that are equally precise and reliable. (X rated 1 employee, and his counts the same as Y who rated 500?) Now, if there are hundreds of respondents, who were all socialized to the same ideal, *that* is useful information to work with. However, I imagine that these numbers could easily be collected from one guy in one state, and three others in the other. If that was so, any comparison you imagine is based on faith, until you add more information to show that the raters mean the same thing. And that, I think, is a much, MUCH bigger problem, than whether the raters conceive of unequal "intervals". > > Can anyone comment on the validity of using averages to report on > categorical data? Or point me to reference sources which would help > clarify the issue? For my points: any book on experimental design. -- Rich Ulrich, [EMAIL PROTECTED] http://www.pitt.edu/~wpilib/index.html = Instructions for joining and leaving this list and remarks about the problem of INAPPROPRIATE MESSAGES are available at http://jse.stat.ncsu.edu/ =
New ways of doing statistics!
Hi! I would like to make an announcement about a new statistical package that has been released for Delphi developers. There are many enviroments to do proffesional level statistics and Borland Delphi compiler, just joined the group See here: www.dewresearch.com An exe demo is also available, so that even those who do not have Delphi, can see what a RAD tool can do for statistics. Best regards! Janz. = Instructions for joining and leaving this list and remarks about the problem of INAPPROPRIATE MESSAGES are available at http://jse.stat.ncsu.edu/ =
RE: Presenting results of categorical data?
Nolan Madson writes: >I have a data set of answers to questions on employee performance. >The answers available are: > >Exceeded Expectations >Met Expectations >Did Not Meet Expectations > >The answers can be assigned weights of 3,2,1 (Exceeded, Met, Did Not >Meet). > >One of my colleagues says that it is not valid to average categorical >data such as this. This is one of those debates that almost takes on a religious overtone. I would not be terribly offended to see a mean reported for this data, though the gap between "Met" and "Did Not Meet" is typically a larger gap than between "Met" and "Exceeded". The "Did Not Meet" rating is typically reserved for those people that a supervisor would like to fire. Other people get quite upset about this, though. You could get some buy-in to an average by first asking any critic what their grade point average was. Unless they respond with a statement like "73% of my grades were B or higher" you have them in a contradictory response. The grade point average is an average of data that is clearly ordinal and where (in my humble opinion) the gap between a "D" and an "F" is much larger than the gap between an "A" and a "B". If a grade point average is considered valid then you might ask how would it differ from the above employee rating scale. Of course, you should read what Deming and others say about employee rating systems, but that is a topic for a different email. If I were doing this project, I would try to get a sense of the folks who will be reading your report. Are they nit pickers? If so, avoid an issue like averaging ordinal data that would give them an extra nit to pick at. Or are they people who despise details and yearn for simplicity. Then use an average which provides a single numeric summary that has a simple (perhaps overly simple) interpretation. And I wouldn't put a lot of time into this problem, since that would tend to add legitimacy to a system (numerical rating of employees) that deserves no legitimacy. Steve Simon, Steve Simon, [EMAIL PROTECTED], Standard Disclaimer. STATS: STeve's Attempt to Teach Statistics. http://www.cmh.edu/stats Watch for a change in servers. On or around June 2001, this page will move to http://www.childrens-mercy.org/stats = Instructions for joining and leaving this list and remarks about the problem of INAPPROPRIATE MESSAGES are available at http://jse.stat.ncsu.edu/ =
Re: Categorical data Take 2
On 15 Aug 2001 09:57:27 -0700, [EMAIL PROTECTED] (Paul R. Swank) wrote: PRS >"If your going to use discriminant analysis you will need a lot of data and it does assume the predictors are multivariate normal." - well, logistic has to assume (almost) the same thing, almost as strongly, when it has to generate a continuous predictor equation. Logistic is practically no different when the prediction is poor - because there will be no 'overshoot' of what is predicted. Logistic is superior when prediction is quite good, *except* sometimes when prediction is too-good, at 100% or near 100%. (A degenerate likelihood surface means: failure of asymptotic behavior, so tests become less reliable.) PRS > "Generalized linear models would seem best, particularly in the event that you don't know if they are ordinal." With 5 groups, is this something different from discriminant function? PRS >"You can do a multinomial followed by a cummulative logit model to see if the data are approximately ordinal." Or you can do discriminant function and see how the categories line up on the first function, and see whether a second function emerges. D.F. programs give you that, but I don't know whether logistic -- Rich Ulrich, [EMAIL PROTECTED] http://www.pitt.edu/~wpilib/index.html = Instructions for joining and leaving this list and remarks about the problem of INAPPROPRIATE MESSAGES are available at http://jse.stat.ncsu.edu/ =
Re: forecast efficiency
Cristian, Let me set you straight. Whomever did you the disservice of teaching you the DW should be scolded. DW only measures 1 period lag. Box-Jenkins methodology uses the Autocorrelation function and partial correlation function to evaluate all lags. I suggest that you look for the Box-Jenkins text book and then do yourself a favor and do a google search on "Box-Jenkins automatic forecasting systems" to use software to identify your model for you. Tom Rich Ulrich <[EMAIL PROTECTED]> wrote in message news:<[EMAIL PROTECTED]>... > On 13 Aug 2001 07:57:35 -0700, [EMAIL PROTECTED] (Cristian Sava) > wrote: > > [ snip, other questions ] > > > >Now it seems that there are other ways of measuring the forecasting > > efficiency, as well: Mincer - Zarnovitz efficiency and conditional > > efficiency of some forecast f1 with respect to another f2. Could > > someone help me understand what these quantities are about and how to > > compute them? (I looked in all the books I had and on the Web, but I > > could not find any reference to these concepts...). > > www.google.com has only 10 hits, total, for Zarnovitz. > So he is not well known and widely-cited. > > Most of them seem to be about a fellow writing on > (forecasting?) business cycles -- judging from the lines > echoed by the search. So, you can look at those articles, > and look up the references in the articles = Instructions for joining and leaving this list and remarks about the problem of INAPPROPRIATE MESSAGES are available at http://jse.stat.ncsu.edu/ =
Re: Optimal bin size for fitting histogram to normal pdf?
Date: Tue, 14 AUG 2001 16:27:11 +1000
From: Hong Ooi <[EMAIL PROTECTED]>
> On 13 Aug 2001 18:59:10 -0700, [EMAIL PROTECTED] (David
> Goldsmith) wrote:
>
> >Aloha! I'm fitting theoretically normally distributed data, of widely
> >differing sample sizes, to Gaussians by histograming it and then using an
> >"off-the-shelf", third-party IDL routine. Obviously, the "goodness" of
> >fit, as measured by the mse, is some function of the bin size used to
> >create the histogram. Some numerical experiments I've run using IDL's
> >pseudo-normal-random number generator and "sample" sizes from 10^2 to
> >10^6.5 indicate that the "best" (that which minimizes the mse) bin size
> >(expressed as a multiple of the sample standard deviation) vs. log(sample
> >size) function is oscillatory, non-periodic. I was hoping for
> >monotonicity so that I could create either a formula or at least a table
> >for this function; not having that, I used the observation that the values
> >seem to be bounded by 0.25 and 0.3 sigma, and, despite it being below any
> >actually observed value, chose 0.25 for "psychological" reasons.
> >Unfortunately, this choice is not working uniformly well, (which actually
> >is not surprising given that the observed "good" range is about 20% of
> >this value). My question for these groups is, does anyone know of any
> >theoretical results on this topic? Thanks,
>
> A standard result is that in terms of minimising MISE, the optimal binwidth
> for a histogram is O(n^{-1/3}), where n is the sample size. For normally
> distributed data, the formula is 3.491 x sigma x n^{-1/3}.
Use your favorite search engines to look up Sturges rule.
> That said, if you know your data really is normally distributed, why do you
> need to fit a histogram anyway? The sample mean and variance give you the
> best possible estimate of the true density, without any need to use
> histograms or smoothers.
If you are trying to develop a technique to estimate confidence levels for
a Gaussian hypothesis test, forget histograms. Sort the samples,
determine the emperical CDF and apply a statistical test like K-S or
Anderson-Darling.
Greg (Brown '62)
Hope this helps.
Gregory E. Heath [EMAIL PROTECTED] The views expressed here are
M.I.T. Lincoln Lab (781) 981-2815not necessarily shared by
Lexington, MA(781) 981-0908(FAX) M.I.T./LL or its sponsors
02420-9185, USA
=
Instructions for joining and leaving this list and remarks about
the problem of INAPPROPRIATE MESSAGES are available at
http://jse.stat.ncsu.edu/
=
