Re: Means of semantic differential scales
At 09:51 AM 2/28/02 -0800, Jay Tanzman wrote: >I partially did this, insofar as I ran Pearson and Spearman correlations >between >several of the scales and, not surprisingly, the two correlation coefficients >and their p-values were similar. <<<<< that issue is entirely a separate >one since the rank order FORMULA was derived from the pearson ... > Dr. Kim was not impressed. > >-Jay i hate to ask this question but, what the heck, spring break is near so i will if your boss, dr. kim??? ... seems so knowledgeable about what the data are and what is and is not appropriate to do with the data, why is not dr. kim doing the analysis? this reminds me of assigning a task to someone and, doing so much micro-managing that ... it would have been better off doing it oneself ... >= >Instructions for joining and leaving this list, remarks about the >problem of INAPPROPRIATE MESSAGES, and archives are available at > http://jse.stat.ncsu.edu/ >= Dennis Roberts, 208 Cedar Bldg., University Park PA 16802 WWW: http://roberts.ed.psu.edu/users/droberts/drober~1.htm AC 8148632401 = Instructions for joining and leaving this list, remarks about the problem of INAPPROPRIATE MESSAGES, and archives are available at http://jse.stat.ncsu.edu/ =
Re: Means of semantic differential scales
> "Simon, Steve, PhD" wrote: > > Jay Tanzman got chewed out by his boss for averaging a 7 point ordinal scale. > Generally it is not a good idea to argue with your boss, but perhaps you might > ask what was the grade point average that he or she received in college. When > you hear the response, then ask if the grading scale A, B, C, D, F is ordinal > or interval. I'm going to ask him. > A possible compromise is to model the data as if it were interval and then > model it as if it were ordinal. If the two models are reasonably similar, > good. If they differ, that is still good, as it allows you to then explore why > the two models differ. I partially did this, insofar as I ran Pearson and Spearman correlations between several of the scales and, not surprisingly, the two correlation coefficients and their p-values were similar. Dr. Kim was not impressed. -Jay = Instructions for joining and leaving this list, remarks about the problem of INAPPROPRIATE MESSAGES, and archives are available at http://jse.stat.ncsu.edu/ =
RE: Means of semantic differential scales
Title: RE: Means of semantic differential scales Jay Tanzman got chewed out by his boss for averaging a 7 point ordinal scale. Generally it is not a good idea to argue with your boss, but perhaps you might ask what was the grade point average that he or she received in college. When you hear the response, then ask if the grading scale A, B, C, D, F is ordinal or interval. A possible compromise is to model the data as if it were interval and then model it as if it were ordinal. If the two models are reasonably similar, good. If they differ, that is still good, as it allows you to then explore why the two models differ. Steve Simon, [EMAIL PROTECTED], Standard Disclaimer. The STATS web page has moved to http://www.childrens-mercy.org/stats
Re: Means of semantic differential scales
On 27 Feb 2002 15:01:24 -0800, [EMAIL PROTECTED] (Dennis Roberts) wrote: >At 01:39 PM 2/27/02 -0600, Jay Warner wrote: > >> > > >> > >Not stressful 1__ 2__ 3__ 4__ 5__ 6__ 7__ Very stressful > >just out of curiosity ... how many consider the above to be an example of a >bipolar scale? > >i don't > >now, if we had an item like: > >sad happy >1 . 7 > >THEN the mid point becomes much more problematic ... > >since being a 4 ... is neither a downer nor upper The bipolar adjectives in Mr. Warner's example might be a tad "fuzzy" IMHO. What is a clear antonym for "stressful"? "Pacified"? "Carefree"? I noted same in my original response to his query. Your item "sad...happy" appears more like what Osgood et al had in mind. "GoodBad," " Hot...Cold, " for example, are clearcut bipolars. If one wants to force an opinion one way or another, then display an even numbered scale. If the investigator wants the "neutral" opinion then make the scale odd numbered. To me the semantic differential is only a Likert Scale without the glitter :-)) I think his supervisor more than likely, however, was concerned about computing means with ordinal data. Perhaps, arguments can be made for both ordinal and interval usage depending on the intent of the research. Some semantic differential instruments I have seen in the past have no printed numerical scale at all. The respondent places a check mark along a horizontally gradated continuum. The researcher then assigns an appropriate score. vis a vis the check mark. Usually bipolar adjective items are randomly assigned, i.e., "good" responses are not all on one side of the document. Supposedly, the respondent can't simply "halo" the concept being evaluated. = Instructions for joining and leaving this list, remarks about the problem of INAPPROPRIATE MESSAGES, and archives are available at http://jse.stat.ncsu.edu/ =
Re: Means of semantic differential scales
DMR, I should have read your previous posting more carefully. I have now had coffee. >Not stressful 1__ 2__ 3__ 4__ 5__ 6__ 7__ Very stressful is a question that has an extent response format. The cognitive schema the response format tries to invoke might be reinforced by anchoring with zero for not at all. To me the low end is zero rather than anti-stressful. In some fields the above might be used as an item in a scale. As in your example, the 16pf uses a series of items to produce bipolar scales. Some concepts make no sense as bipolar scales. Ability, achievement, etc. have no cognitive opposites. Even preferences and attitudes are necessarily measured with opposites. Bem & associates made much of the fact that adaption to gender expectations should be represented with two dimensions (analogous to longitude to and latitude) so that it took 2 variables to adequately represent that concept. The degre of having attributes popularly considered characteristic of masculinity were construed as orthogonal to the degree of having attributes popularly considered characteristic of femininity. With regard to the original question, in my opinion, there is nothing automatically incorrect about getting means on such variables. If the purpose is to compare groups, it is more important to be sure to use the same ruler, than it is to worry whether it is a rubber ruler. Dennis Roberts wrote: > At 01:39 PM 2/27/02 -0600, Jay Warner wrote: > > > > > > > > >Not stressful 1__ 2__ 3__ 4__ 5__ 6__ 7__ Very stressful > > just out of curiosity ... how many consider the above to be an example of a > bipolar scale? > > i don't = Instructions for joining and leaving this list, remarks about the problem of INAPPROPRIATE MESSAGES, and archives are available at http://jse.stat.ncsu.edu/ =
Re: Means of semantic differential scales
I would consider it a unipolar extent scale. Maybe the visual anchor should be 0 to 6 to aid association with the number line concept. Dennis Roberts wrote: > At 01:39 PM 2/27/02 -0600, Jay Warner wrote: > > > > > > > > >Not stressful 1__ 2__ 3__ 4__ 5__ 6__ 7__ Very stressful > > just out of curiosity ... how many consider the above to be an example of a > bipolar scale? > > i don't > > = Instructions for joining and leaving this list, remarks about the problem of INAPPROPRIATE MESSAGES, and archives are available at http://jse.stat.ncsu.edu/ =
Re: Means of semantic differential scales
At 01:39 PM 2/27/02 -0600, Jay Warner wrote: > > > > > >Not stressful 1__ 2__ 3__ 4__ 5__ 6__ 7__ Very stressful just out of curiosity ... how many consider the above to be an example of a bipolar scale? i don't now, if we had an item like: sad happy 1 . 7 THEN the mid point becomes much more problematic ... since being a 4 ... is neither a downer nor upper now, a quick search found info from ncs about the 16pf personality scale ... it shows 16 BIpolar dimensions as: Bipolar Dimensions of Personality Factor A Warmth (Cool vs Warm) Factor B Intelligence (Concrete Thinking vs Abstract Thinking) Factor C Emotional Stability (Easily Upset vs Calm) Factor E Dominance (Not Assertive vs Dominant) Factor F Impulsiveness (Sober vs Enthusiastic) Factor G Conformity (Expedient vs Conscientious) Factor H Boldness (Shy vs Venturesome) Factor I Sensitivity (Tough-Minded vs Sensitive) Factor L Suspiciousness (Trusting vs Suspicious) Factor M Imagination (Practical vs Imaginative) Factor N Shrewdness (Forthright vs Shrewd) Factor O Insecurity (Self-Assured vs Self-Doubting) Factor Q1 Radicalism (Conservative vs Experimenting) Factor Q2 Self-Sufficiency (Group-Oriented vs Self-Sufficient) Factor Q3 Self-Discipline (Undisciplined vs Self-Disciplined) Factor Q4 Tension (Relaxed vs Tense) let's take the one ... shy versus venturesome ... now, we could make a venturesome scale by itself ... 0 venturesomeness .. (up to) very venturesome 7 does 0 = shy seems like if the answer is no ... then we might have a bipolar scale ... if the answer is yes ... then we don't > It could be the use of the particular bipolars > > "not stressful" and "very stressful." >= Dennis Roberts, 208 Cedar Bldg., University Park PA 16802 WWW: http://roberts.ed.psu.edu/users/droberts/drober~1.htm AC 8148632401 = Instructions for joining and leaving this list, remarks about the problem of INAPPROPRIATE MESSAGES, and archives are available at http://jse.stat.ncsu.edu/ =
Re: Means of semantic differential scales
I am humbled by the insight & background knowledge expressed by Mssrs. Williams and McLean, not to mention the string of others. My lack of academic experince in the subject matter is painfully clear. Now to see if I can find Osgood et al. When I consider how many research projects and social/political actions depend on survey responses for their information, the need for this level of 'prethinking' becomes all the more necessary. Jay "J. Williams" wrote: > On Mon, 25 Feb 2002 15:17:55 -0800, Jay Tanzman <[EMAIL PROTECTED]> > wrote: > > >I just got chewed out by my boss for modelling the means of some 7-point > >semantic differential scales. The scales were part of a written, > >self-administered questionnaire, and were laid out like this: > > > >Not stressful 1__ 2__ 3__ 4__ 5__ 6__ 7__ Very stressful > > > >So, why or why not is it kosher to model the means of scales like this? > > > >-Jay > > You can check it out by reading the pioneers of the semantic > differential scale. Osgood, Suci, and Tannenbaum are the authors of > "Measurement of Meaning" which now is published in paperback by the > University of Illinois Press, Oct. 1990. It may be your boss is a > stickler on what constitutes a true interval scale. It could be > he/she wants no middle value score - that way respondents must tilt > toward a yea or nay. It could be the use of the particular bipolars > "not stressful" and "very stressful." Why not use stressful and not > stressful? What is "very" stressful? By reading the Osgood et al > text, you can find many nifty ideas and variations for using the > semantic differential scale. Like the Likert Scale, I suppose it is > arguably an ordinal scale. But, there are lots of statistical tools > you could employ using rankings, medians, etc. Like the Likert Scale > devotees, there are those who nevertheless use means as the measure > of central tendency with semantic differential instruments. Good > luck. > > = > Instructions for joining and leaving this list, remarks about the > problem of INAPPROPRIATE MESSAGES, and archives are available at > http://jse.stat.ncsu.edu/ > = -- Jay Warner Principal Scientist Warner Consulting, Inc. North Green Bay Road Racine, WI 53404-1216 USA Ph: (262) 634-9100 FAX: (262) 681-1133 email: [EMAIL PROTECTED] web: http://www.a2q.com The A2Q Method (tm) -- What do you want to improve today? = Instructions for joining and leaving this list, remarks about the problem of INAPPROPRIATE MESSAGES, and archives are available at http://jse.stat.ncsu.edu/ =
Re: Means of semantic differential scales
Jay Tanzman wrote: > > Jay Warner wrote: > > > > Jay Tanzman wrote: > > > > > I just got chewed out by my boss for modelling the means of some 7-point > > > semantic differential scales. The scales were part of a written, > > > self-administered questionnaire, and were laid out like this: > > > > > > Not stressful 1__ 2__ 3__ 4__ 5__ 6__ 7__ Very stressful > > > > > > So, why or why not is it kosher to model the means of scales like this? > > > > > > -Jay > > My boss's objection was that he believes "categorically" (sorry) that semantic > differential scales are ordinal. > > > 1)Why do you think the scale is interval data, and not ordinal or > > categorical? > > Why would anyone think it is ordinal and not interval? Most of the scales were > measuring abstract, subjective constructs, such as empathy and satisfaction, for > which there is no underlying physical or biological measurement. Why not, then, > _define_ degree of empathy as the subjects' rating on a 1-to-7 scale? > Why not indeed?! Of course you can do this - and in fact you are doing this. The question is really - what properties should this variable possess in order that it is meaningful - that is, that it reflects 'reality' meaningfully. If it does not do this, then whatever conclusions you come to about your variable are of no use whatsoever. It is certainly true that your variable is ordinal. Is it more than this? It is extremely unlikely that it is fully numeric (that is, 'interval') because the difference between 1 and 2 is unlikely to have the same meaning as the difference between 4 and 5. You cannot simply define these differences to be equal - you need your variable to reflect reality! However, it is probable that the scale is 'reasonably numeric', so the assumption that the variable is interval may be reasonable. But this will be a model, using a number of assumptions - as all these things are. It is important that you recognise this modelling aspect of your data definition. Regards, Alan -- Alan McLean ([EMAIL PROTECTED]) Department of Econometrics and Business Statistics Monash University, Caulfield Campus, Melbourne Tel: +61 03 9903 2102Fax: +61 03 9903 2007 = Instructions for joining and leaving this list, remarks about the problem of INAPPROPRIATE MESSAGES, and archives are available at http://jse.stat.ncsu.edu/ =
Re: Means of semantic differential scales
> > > 2. Perhaps more likely, your boss may have learned > > (wrongly?) that parametric stats should not be done unless scales > > of measurement are at least interval in quality. > > I don't know if his objection was to parametric statistics per se, but he did > object to calculating means on these data, which he believes are only ordinal. > > > Search on google > > for people like John? Gaito and S.S. Stevens and for phrases like > > "scales of measurement" and "parametric statistics." > > Thanks. Will do. > Or, do an Advanced search with groups.google among the sci.stat.* groups for < Stevens, measurement >. I think that would find earlier discussions and some references. As I recall it, no one who pretended to know much would have sided with your boss. The firmness of Stevens's categories was strongly challenged by the early 1950s. In particular, there was Frederick Lord's ridiculing parable of the football jerseys. (Naturally, psychology departments taught the subject otherwise, for quite a while longer.) Conover, et al., took a lot of the glory out of 'nonparametric tests' by showing that you can't gain much from rank-order transformations, compared to any decent scaling. That was in an article of 1980 or thereabouts. I may have seen a 'research manual' dated as recent as 1985 that still favored using rank-statistics with Likert-scaled items. I am curious as to what more recent endorsements might exist, in any textbooks at all, or in papers by statisticians. -- Rich Ulrich, [EMAIL PROTECTED] http://www.pitt.edu/~wpilib/index.html = Instructions for joining and leaving this list, remarks about the problem of INAPPROPRIATE MESSAGES, and archives are available at http://jse.stat.ncsu.edu/ =
Re: Means of semantic differential scales
At 08:18 AM 2/26/02 -0800, Jay Tanzman wrote: > > > > > > Not stressful 1__ 2__ 3__ 4__ 5__ 6__ 7__ Very stressful these contain more information than simply ordinality ... they give you some indication of amount of stress too differentiate this sort of item and response from: rank order your preferences for the following foods: steak ___ ... 1 veal ... 2 chicken ... 4 fish ... 5 pork ... 3 and, assume it says to put 1 for the top 1 ... and 5 for the low one so, i do as above both CAN be thought of as ordering scales ... but, there is definitely MORE information in the not stressful to very stressful item and responses the end points of the 1 to 7 scale DO have meaning ... in terms of ABSOLUTE quantities that is not so for the food orderings ... can we infer that i don't like fish since i ranked it 5 and DO like steak since i ranked it one??? NOT necessarily there is a fundamental difference in the information you can extract from each of the examples above i see nothing inherently wrong with finding means on items like the stress item ... since means close to 1 or 7 ... do have some underlying referent to quantity of stress ... one cannot say that about the food preferences in terms of some underlying absolute liking or disliking of the foods Dennis Roberts, 208 Cedar Bldg., University Park PA 16802 WWW: http://roberts.ed.psu.edu/users/droberts/drober~1.htm AC 8148632401 = Instructions for joining and leaving this list, remarks about the problem of INAPPROPRIATE MESSAGES, and archives are available at http://jse.stat.ncsu.edu/ =
Re: Means of semantic differential scales
i think we are all missing the main point if you have a number of these items where, your goal (perhaps) is to SUM them together in some way ... where one end represents low amounts of the "thing" presented and the other end represents large amounts of the thing presented ... then ACROSS items ... the issue is do Ss tend to respond at the low end or the high end? i really don't care if the exact scale IS interval or interpreted by Ss as such ... the main thing is how do they respond across a set of items? whether or not these data or scales are interval or not, the MEAN has meaning ... excuse the pun ... i am willing to bet that those Ss who produce mean values close to 1 below are not experiencing any serious stress ... whereas those Ss whose means are close to 6 or 7 ... are now, does that mean i know precisely what they are thinking/feeling? of course not but, it is plenty good enough to get a good idea of variation across Ss on these items or dimensions i really don't see what the big fuss is At 08:10 AM 2/26/02 -0800, Jay Tanzman wrote: >Jay Warner wrote: > > > > Jay Tanzman wrote: > > > > > I just got chewed out by my boss for modelling the means of some 7-point > > > semantic differential scales. The scales were part of a written, > > > self-administered questionnaire, and were laid out like this: > > > > > > Not stressful 1__ 2__ 3__ 4__ 5__ 6__ 7__ Very stressful Dennis Roberts, 208 Cedar Bldg., University Park PA 16802 WWW: http://roberts.ed.psu.edu/users/droberts/drober~1.htm AC 8148632401 = Instructions for joining and leaving this list, remarks about the problem of INAPPROPRIATE MESSAGES, and archives are available at http://jse.stat.ncsu.edu/ =
Re: Means of semantic differential scales
Jay Warner wrote: > > Jay Tanzman wrote: > > > I just got chewed out by my boss for modelling the means of some 7-point > > semantic differential scales. The scales were part of a written, > > self-administered questionnaire, and were laid out like this: > > > > Not stressful 1__ 2__ 3__ 4__ 5__ 6__ 7__ Very stressful > > > > So, why or why not is it kosher to model the means of scales like this? > > > > -Jay My boss's objection was that he believes "categorically" (sorry) that semantic differential scales are ordinal. > 1)Why do you think the scale is interval data, and not ordinal or > categorical? Why would anyone think it is ordinal and not interval? Most of the scales were measuring abstract, subjective constructs, such as empathy and satisfaction, for which there is no underlying physical or biological measurement. Why not, then, _define_ degree of empathy as the subjects' rating on a 1-to-7 scale? > If interval, the increments between the levels are more or > less equal. If ordinal we know they are sequential, but have no idea how > far apart each pair is. Categorical means there is no relationship between > them - 4 is not greater than 3 - it's only different. > > Some people use a response of 4 to mean 'no response' as well as 'no > opinion' and 'neutral opinion.' sorry, these are not intervals. > > 2)Is it possible for a respondent to come back with 2.5? If so, they > think it is interval data, regardless of your opinion. Would you throw out > a response of 2.5, or would you enter it in your dataset as 2.5? If the > latter, you think it is interval, also. An obscure corollary to the Law of Large Numbers is that, in a self-administered questionnaire, the probability that some individual will either write in some-number-point-five (or, equivalently, check two adjacent numbers) approaches 1 as N increases without bound. I would have no theoretical objection to them doing that on this survey. > 3)What makes you think the scale is linear (equal intervals)? My boss's argument that it is not interval is that subjects don't necessarily treat it that way. That is, they don't treat the difference between 1 and 2 as the same as, say, between 3 and 4. My feeling is that there is no natural unit of, say, satisfaction, so why not define a unit of satisfaction as the rating on the scale. > It ain't > - since respondents can't go below 1 or above 7 . Well, maybe 0 and 8, but > the point is the same. If you must, make a transformation (arc-sine for > starters) to make it more 'linear' and more likely to contain Normal dist. > data. The scale can have limits and still be interval. The amount of water in an 8 oz. glass is constrained to be between 0 and 8, but ounces on water in the glass would still be interval data. > 4)Why might the respondents use the same increments that you think > exist, or the same as other respondents? If there is some way you can > 'anchor' at end points or mid point, you will get much more informative > data. I mean, what is 'very stressful' to you? To me? to anyone? I don't think it matters. What is 'very stressful' to the individual respondent is what is important. For one thing, we were testing hypotheses about the effects of alternative programs on these subjective outcomes. As long as there was no association between how respondents interpret the scales and which program they attended, I don't see how differences in scale interpretation could affect the results; there would be no confounding. > 5)In cases where I have been able to anchor firmly, and in some where I > haven't, I find that treating the scale as incremental data work just fine, > thank you. I agree. Assuming that the data, which consist of the numbers 1 to 7, are interval in the absence of evidence to the contrary seems like a pretty mild assumption to me. Furthermore, even if they are not interval, treating them as such would seem unlikely to cause any great bias in the results. > As soon as you compute an average of responses on this scale, > you have done just that. If you restrict yourself to categorical analysis > for frequencies between categories, you have avoided that assumption. And > you have far less to say about the data, as well. Treating this data as categorical would have led to very sparse data. Ordinal logistic regression would have been messy because I would had to collapse categories, and this defeats the purpose of having the categories in the first place. Treating the data as interval allowed me to evaluate the treatments and their interactions using multiple linear regression, though, possibly, I could have done this on the ranks of t
Re: Means of semantic differential scales
jim clark wrote: > > Hi > > On Mon, 25 Feb 2002, Jay Tanzman wrote: > > > I just got chewed out by my boss for modelling the means of some 7-point > > semantic differential scales. The scales were part of a written, > > self-administered questionnaire, and were laid out like this: > > > > Not stressful 1__ 2__ 3__ 4__ 5__ 6__ 7__ Very stressful > > > > So, why or why not is it kosher to model the means of scales like this? [snip] > 2. Perhaps more likely, your boss may have learned > (wrongly?) that parametric stats should not be done unless scales > of measurement are at least interval in quality. I don't know if his objection was to parametric statistics per se, but he did object to calculating means on these data, which he believes are only ordinal. > Search on google > for people like John? Gaito and S.S. Stevens and for phrases like > "scales of measurement" and "parametric statistics." Thanks. Will do. -Jay = Instructions for joining and leaving this list, remarks about the problem of INAPPROPRIATE MESSAGES, and archives are available at http://jse.stat.ncsu.edu/ =
Re: Means of semantic differential scales
"J. Williams" wrote: > > On Mon, 25 Feb 2002 15:17:55 -0800, Jay Tanzman <[EMAIL PROTECTED]> > wrote: > > >I just got chewed out by my boss for modelling the means of some 7-point > >semantic differential scales. The scales were part of a written, > >self-administered questionnaire, and were laid out like this: > > > >Not stressful 1__ 2__ 3__ 4__ 5__ 6__ 7__ Very stressful > > > >So, why or why not is it kosher to model the means of scales like this? > > > >-Jay > > You can check it out by reading the pioneers of the semantic > differential scale. Osgood, Suci, and Tannenbaum are the authors of > "Measurement of Meaning" which now is published in paperback by the > University of Illinois Press, Oct. 1990. Thanks. I'll do that. I think one of the above authors also has a website, though, yesterday it crashed my Browser. Then again, my browser was Netscape... > It may be your boss is a > stickler on what constitutes a true interval scale. Yes, that is it. See my response to Jay Warner for the details. -Jay = Instructions for joining and leaving this list, remarks about the problem of INAPPROPRIATE MESSAGES, and archives are available at http://jse.stat.ncsu.edu/ =
Re: Means of semantic differential scales
Hi On Mon, 25 Feb 2002, Jay Tanzman wrote: > I just got chewed out by my boss for modelling the means of some 7-point > semantic differential scales. The scales were part of a written, > self-administered questionnaire, and were laid out like this: > > Not stressful 1__ 2__ 3__ 4__ 5__ 6__ 7__ Very stressful > > So, why or why not is it kosher to model the means of scales like this? Two possibilities suggest themselves (there are probably more), although somewhat unclear to me what you mean by "modelling the means." 1. You are aggregating across items that the boss thinks should be analyzed separately, either because they measure different constructs or because some are reverse-worded? 2. Perhaps more likely, your boss may have learned (wrongly?) that parametric stats should not be done unless scales of measurement are at least interval in quality. Search on google for people like John? Gaito and S.S. Stevens and for phrases like "scales of measurement" and "parametric statistics." This debate surfaces now and then, so there are probably things in various archives as well. Best wishes Jim James M. Clark (204) 786-9757 Department of Psychology(204) 774-4134 Fax University of Winnipeg 4L05D Winnipeg, Manitoba R3B 2E9 [EMAIL PROTECTED] CANADA http://www.uwinnipeg.ca/~clark = Instructions for joining and leaving this list, remarks about the problem of INAPPROPRIATE MESSAGES, and archives are available at http://jse.stat.ncsu.edu/ =
Re: Means of semantic differential scales
of course, to be fair to the first jay .. could be simply that his boss did not like semantic diff. scales ... AND, for none of the reasons the second jay below said ... it would be helpful if the first jay could give us some further info on why his boss was so ticked off ... At 09:39 PM 2/25/02 -0600, Jay Warner wrote: >Jay Tanzman wrote: > > > I just got chewed out by my boss for modelling the means of some 7-point > > semantic differential scales. The scales were part of a written, > > self-administered questionnaire, and were laid out like this: > > > > Not stressful 1__ 2__ 3__ 4__ 5__ 6__ 7__ Very stressful > > > > So, why or why not is it kosher to model the means of scales like this? > >= >Instructions for joining and leaving this list, remarks about the >problem of INAPPROPRIATE MESSAGES, and archives are available at > http://jse.stat.ncsu.edu/ >= = Instructions for joining and leaving this list, remarks about the problem of INAPPROPRIATE MESSAGES, and archives are available at http://jse.stat.ncsu.edu/ =
Re: Means of semantic differential scales
Jay Tanzman wrote: > I just got chewed out by my boss for modelling the means of some 7-point > semantic differential scales. The scales were part of a written, > self-administered questionnaire, and were laid out like this: > > Not stressful 1__ 2__ 3__ 4__ 5__ 6__ 7__ Very stressful > > So, why or why not is it kosher to model the means of scales like this? > > -Jay 1)Why do you think the scale is interval data, and not ordinal or categorical? If interval, the increments between the levels are more or less equal. If ordinal we know they are sequential, but have no idea how far apart each pair is. Categorical means there is no relationship between them - 4 is not greater than 3 - it's only different. Some people use a response of 4 to mean 'no response' as well as 'no opinion' and 'neutral opinion.' sorry, these are not intervals. 2)Is it possible for a respondent to come back with 2.5? If so, they think it is interval data, regardless of your opinion. Would you throw out a response of 2.5, or would you enter it in your dataset as 2.5? If the latter, you think it is interval, also. 3)What makes you think the scale is linear (equal intervals)? It ain't - since respondents can't go below 1 or above 7 . Well, maybe 0 and 8, but the point is the same. If you must, make a transformation (arc-sine for starters) to make it more 'linear' and more likely to contain Normal dist. data. 4)Why might the respondents use the same increments that you think exist, or the same as other respondents? If there is some way you can 'anchor' at end points or mid point, you will get much more informative data. I mean, what is 'very stressful' to you? To me? to anyone? Perhaps you are evaluating how people respond to specific scenarios with their impression of anticipated stress. In which case, the strength of 'very' is at issue, and perhaps you can argue that it is what you are measuring. (remember the old maps: there be dragons). When I sit down with a client to work out an experimental design for a project, one might call this highly stressful. I am in full control of the alternatives and options, so to me it is great fun, and very invigorating. the situation is far from 'Not stressful' - it is not opposite of 'stressful.' I know my muscles have been stressed, because it is also very tiring. so what might be 'stressful'? Is that worked out with your respondents beforehand? 5)In cases where I have been able to anchor firmly, and in some where I haven't, I find that treating the scale as incremental data work just fine, thank you. As soon as you compute an average of responses on this scale, you have done just that. If you restrict yourself to categorical analysis for frequencies between categories, you have avoided that assumption. And you have far less to say about the data, as well. Cheers, Jay -- Jay Warner Principal Scientist Warner Consulting, Inc. North Green Bay Road Racine, WI 53404-1216 USA Ph: (262) 634-9100 FAX: (262) 681-1133 email: [EMAIL PROTECTED] web: http://www.a2q.com The A2Q Method (tm) -- What do you want to improve today? = Instructions for joining and leaving this list, remarks about the problem of INAPPROPRIATE MESSAGES, and archives are available at http://jse.stat.ncsu.edu/ =
Re: Means of semantic differential scales
On Mon, 25 Feb 2002 15:17:55 -0800, Jay Tanzman <[EMAIL PROTECTED]> wrote: >I just got chewed out by my boss for modelling the means of some 7-point >semantic differential scales. The scales were part of a written, >self-administered questionnaire, and were laid out like this: > >Not stressful 1__ 2__ 3__ 4__ 5__ 6__ 7__ Very stressful > >So, why or why not is it kosher to model the means of scales like this? > >-Jay You can check it out by reading the pioneers of the semantic differential scale. Osgood, Suci, and Tannenbaum are the authors of "Measurement of Meaning" which now is published in paperback by the University of Illinois Press, Oct. 1990. It may be your boss is a stickler on what constitutes a true interval scale. It could be he/she wants no middle value score - that way respondents must tilt toward a yea or nay. It could be the use of the particular bipolars "not stressful" and "very stressful." Why not use stressful and not stressful? What is "very" stressful? By reading the Osgood et al text, you can find many nifty ideas and variations for using the semantic differential scale. Like the Likert Scale, I suppose it is arguably an ordinal scale. But, there are lots of statistical tools you could employ using rankings, medians, etc. Like the Likert Scale devotees, there are those who nevertheless use means as the measure of central tendency with semantic differential instruments. Good luck. = Instructions for joining and leaving this list, remarks about the problem of INAPPROPRIATE MESSAGES, and archives are available at http://jse.stat.ncsu.edu/ =
Means of semantic differential scales
I just got chewed out by my boss for modelling the means of some 7-point semantic differential scales. The scales were part of a written, self-administered questionnaire, and were laid out like this: Not stressful 1__ 2__ 3__ 4__ 5__ 6__ 7__ Very stressful So, why or why not is it kosher to model the means of scales like this? -Jay = Instructions for joining and leaving this list, remarks about the problem of INAPPROPRIATE MESSAGES, and archives are available at http://jse.stat.ncsu.edu/ =
Re: Simulating T tests for Likert scales
Will, I gotta reply to this one! I've done this type of thing a number of times. Will Hopkins wrote: > I have an important (for me) question, but first a preamble and > hopefully some useful info for people using Likert scales. > > A week or so ago I initiated a discussion about how non-normal the > residuals have to be before you stop trusting analyses based on > normality. Someone quite rightly pointed out that it depends on the > sample size, because the sampling distribution of almost every > statistic derived from a variable with almost any distribution is near > enough to normal for a large enough sample, thanks to the central > limit theorem. Therefore you get believable confidence limits from t > statistics. The distribution of the average of 12 observations, taken from a 'saw tooth' population, is about 1 significant line width away from a normal population. n, the sample size, doesn't have to be very big. > > But how non-normal, and how big a sample? I have been doing > simulations to find out. I've limited the simulations to t tests for > Likert scales with only a few levels, because these crop up often in > research, and Likert-scale variables with responses stacked up at one > end are not what you call normally distributed. Yes, I know you can > and maybe should analyze these with logistic regression, but it's hard > work for statistically challenged research students, and the outcomes > (odds ratios) are hard for all but statisticians to understand. > Scoring the levels with integers and working out averages is so much > easier to do and interpret. > > My simulations have produced some seemingly amazingly good results. > For example, with a 3-point scale with values of 1, 2 and 3, samples > of as few as 10 in each of two groups give accurate confidence > intervals for the difference in the means of the groups when both > means are ~2.0 (i.e. in the middle) and SDs are ~0.7 (i.e. the > majority of observations on 2, with a reasonable number on 1 and 3). > They are still accurate even when one of the groups is stacked up at > one end with a mean of 2.6 (and SD ~0.5). If both means are stacked > up at one or either end, sample sizes of 20 or more are needed, > depending on how extreme the stacking is. Likert scales with more > than 3 levels work perfectly for anything except responses stacked up > in the same extreme way at either end. Aren't these getting over toward some kind of binary distribution? > Now, my question. Suppose in real life I have a sample of 10 > observations of, say, a 5-point scale scored as 1 to 5. Suppose I get > 1 response on 3, 5 responses on 4 and 4 responses on 5. You have assumed that a response must be integer - i.e., ordinal scale. The best 'resolution' of your scale is, roughly, 20% - one unit in 5. If I knew enough math, I might be able to show what is the least difference in two means that you could use, to demonstrate a difference in those means. For a given sample size. > The mean is therefore 4.3. Suppose the other group is no problem > (e.g., 10 or more responses spread around the middle somewhere). Now, > according to my simulations, it's OK for me to do a t test to get the > confidence limits for the difference, isn't it? Now suppose the first > group was stacked more extremely, with 2 on 4 and 8 on 5. The mean > for this group is now 4.8. According to my simulations, that's too > extreme to apply the t test, with a sample of 10, anyway. Suppose I have 5 coins, weighted so p(heads) = .96. Count a head as 1, a tail as 0. Toss 5 and add up the coins. Multile times. Average: 4.8 Could I use the binary caluclations to determine the sample size requried before the Student 't' and normal dist. could apply? You bet! > Is this the correct way to apply the results of my simulations? I can > see how it could fall over: you could in principle get a sample of > 1x3, 5x4 and 4x5 when the true distribution has a mean of 4.8, but the > chance of that happening is small. > > To put the question in a more general context of simulation: if the > observed sample has a value of the outcome statistic that simulations > show has an accurate confidence interval for the given sample size > when that value is the population value of the statistic, is the > resulting confidence interval accurate? > > Will I'm not clear why you 'give' away information by making your Likert scale into an ordinal value, instead of accepting fractional units, such as 0.5 (2.5, 4.5, etc.). Whenever a survey respondent puts the 'x' mark part way between the box for 'neutral' and 'somewhat agree,' they are trying to t
Re: scales
i think most would agree that the inclusion of something akin to ? on an item scale creates problems ... but, the lack of having that ? does not mean that all problems go away ... because, even in that case ... what does a S do if they truly have not particular leaning one way or the other? unless then leave it blank ... they will put down some response which, of course ... is not accurate for them just like anything else ... the saving grace is to have a reasonable number of items AND hope that "on average" ... then tend to be fairly + or fairly - for, if the average is close to some midpoint value (whether this is an actually spot on the scale or not) ... the interpretation of the score can have its greatest confusion ... one could be quite - for about 1/2 of the items ... and quite + for about 1/2 of the items ... averaging sort of "neutral" across the entire scale ... or, one could put the middle kind of response on almost ALL items ... thus getting a "neutral" kind of average across the entire scale and we all know that these two different response patterns DON'T have the same meaning at all ... the overall problem is the average ... and what IT tells us and what it does not tell us ... At 08:46 AM 1/5/01 -0500, Bob Hayden wrote: >An example might shed some light on one point involved here. Recently >Plymouth State College considered the possibility of arming -- well, >part of the question was WHOM. Many of us refered to them as Campus >Security while they insisted they were Campus Police. = Instructions for joining and leaving this list and remarks about the problem of INAPPROPRIATE MESSAGES are available at http://jse.stat.ncsu.edu/ =
scales
An example might shed some light on one point involved here. Recently Plymouth State College considered the possibility of arming -- well, part of the question was WHOM. Many of us refered to them as Campus Security while they insisted they were Campus Police. Looking at old phone books it appeared the name had changed some years ago. We did not do a formal survey, but one question might be how people felt about this change. In our situation I think most of the responses would be either "I did not know this change took place" or "I have no idea what the change means, other than a change in the label". (Whatever you call them, the folks whose name changed seemed to feel the change was very significant to them.) Such answers would probably be lumped under a "?" choice and they do NOT mean neutrality or "this does not apply to me". Indeed, in this situation relying on a "?" option would tend to blind one to the fact that what might really be appropriate here would be questions like "Prior to the recent controversy on arming, did you know Campus Security had become Campus Police?" "Do you feel you have a clear understanding of what this change means?" -- _ | |Robert W. Hayden | | Work: Department of Mathematics / |Plymouth State College MSC#29 | |Plymouth, New Hampshire 03264 USA | * |fax (603) 535-2943 /| Home: 82 River Street (use this in the summer) | )Ashland, NH 03217 L_/(603) 968-9914 (use this year-round) Map of New[EMAIL PROTECTED] (works year-round) Hampshire http://mathpc04.plymouth.edu (works year-round) The State of New Hampshire takes no responsibility for what this map looks like if you are not using a fixed-width font such as Courier. "Opportunity is missed by most people because it is dressed in overalls and looks like work." --Thomas Edison = Instructions for joining and leaving this list and remarks about the problem of INAPPROPRIATE MESSAGES are available at http://jse.stat.ncsu.edu/ =
comparing dispersion of items with different scales
Sorry for my late reaction. Anyway, thank you for your answers. In contrast to the two big US parties European parties often occupy only a subset of the opinion spectrum. There are small parties that distinguish themselves from e.g. Socialist parties on the one end of the spectrum vs. conservative parties on the other end by choosing a position somewhere in the middle of an issue scale. I am prepared to make the assumption that the L-R-scales or any of the other issue scales are of an interval scale nature. All comparable studies in political science have done the same. (I hear the sighs! These social scientists are far too liberal :) ) Marcelo Jenny Vienna, Austria === This list is open to everyone. Occasionally, less thoughtful people send inappropriate messages. Please DO NOT COMPLAIN TO THE POSTMASTER about these messages because the postmaster has no way of controlling them, and excessive complaints will result in termination of the list. For information about this list, including information about the problem of inappropriate messages and information about how to unsubscribe, please see the web page at http://jse.stat.ncsu.edu/ ===
