Re: over and above
dennis roberts wrote: grading projects for a first assignment REreminds me that ... some students go way above and beyond the call of duty when doing projects ... in my case, they have to download a file ... do some analyses ... and then do some write up of what they found. now, some go to alot to trouble to do very nice documents in a word processor and have gone to fancy extents ... whereas others will use (minitab is the package in this case) minitab ... and do all their work in it (which is possible) and turn in an acceptable document ... say the right things, etc. the QUALITY of the document is not a gradable criterion (in my system ... maybe it should be) ... but, how do any of you deal with this sort of thing ... whether it be a document to be turned in or some other assignment ... some just go WAY over what is required ... of course, penn state has no A+ grade category .. !!! Grading projects is difficult, but it has become easier for me ever since I have adopted the "dimensions" approach. I like to give them about ten "dimensions" to shoot for, and then sum the ten to get the final grade. Some dimensions, like "correct analysis" and "correct interpretations" get a lot more weight. "Professional appearance" is a good dimension to include. After all, if they produce reports in the "real world", they should have a professional appearance. Another dimension I use is "completeness". This one is a double-edged sword - they have to include enough analysis to be convincing, but extraneous "padding" counts against them. I would not complain about having too many good students! It does bring back the issue of "grading on the curve", though. Anyone want to re-start that thread? Peter === This list is open to everyone. Occasionally, less thoughtful people send inappropriate messages. Please DO NOT COMPLAIN TO THE POSTMASTER about these messages because the postmaster has no way of controlling them, and excessive complaints will result in termination of the list. For information about this list, including information about the problem of inappropriate messages and information about how to unsubscribe, please see the web page at http://jse.stat.ncsu.edu/ ===
Re: grading on the curve
Herman Rubin wrote: In article 83umq6$75s$[EMAIL PROTECTED], a [EMAIL PROTECTED] wrote: In article 83ugke$[EMAIL PROTECTED], [EMAIL PROTECTED] says... In article [EMAIL PROTECTED], Rich Ulrich [EMAIL PROTECTED] wrote: On 22 Dec 1999 14:47:38 -0800, [EMAIL PROTECTED] (dennis roberts) wrote: Actually, I see where I might want to be more arbitrary that just changing a cutoff. How do you reward someone who is really trying hard, vs. someone who is smart but is blowing it off? Why should you? The grade should be on knowledge and the ability to use it, not on effort. If somebody is born with the knowledge, he deserves the grade and credit. If someone works full time and cannot do it, he deserves to fail. Hm, just because a student is born with the knowledge, he/she should deserve the grade and credit although he/she didn't do well in the class??? What is the purpose of a grade? The legitimate purposes are to tell the world what the student knows and can do, and to advise the student on the same matter. One can have the latter without the former; I believe in comprehensive examinations to provide information to others. Your comment suggests that our primary job as educators is to rank the students. I disagree. Our primary job to educate the students. The latter purpose (advising) is indeed legitimate. The former (tell the world what they know) is what we use in our current educational system, and its value as a predictive tool is questionable. In the case of the student who slacks off and gets an "A", the predictive use of the "A" is highly questionable. A future employer might see the "A" as indicative of diligence, hard work, when such is clearly not the case. If the student already knows the material and slacks off through my class, I would be happy to tell the world that this student is not someone you want (to hire, to be in your class, to work with). My advice to such a student would be not to take my class, especially if their thought is to receive an easy "A". Related to this discussion is the well-documented low predictive ability of SAT scores. Use of such tools (grades, SAT scores) that have low predictive ability to make decisions that affect individuals' lives amounts to little more than a lottery, mentioned previously in this discussion group by Eric Bohlman. There have been at least three empirical examples presented in the current discussion that suggest that the use of ranking is detrimental - one my example about Texas Instruments, another a study on how ranking stifled creativity in art students (see Steve Simon's post), a third mentioned in Eric Bohlman's post. What empirical evidence is there to the contrary? I respect students who try hard and give their best. I have no respect for smart students who don't live up to their talents. If a student works full time and still can't do it, I'll never ever fail him/her. To me, the most important thing is that you give your best. This might be from the standpoint of socialist ethics, but not from the standpoint of education. Especially if grades are not public knowledge, one is doing a service by failing a student who is unable to grasp the material. Better yet, such a student should be properly advised. Peter
Re: adjusting marks; W. Edwards Deming
Jim Clark wrote: Artificially giving all students (or almost all) the same grade does not minimize variation in the underlying trait, achievement, in this case. It simply hides the variation so that one does not know to what extent one is minimizing differences in achievement, and rewards students for not trying to achieve more than some minimal level. I don't Deming would have said assignment of Pass/Fail should be "artificial". If the student doesn't perform, then of course they shouldn't Pass. He did say, on the other hand, that grading imposes an artificial scarcity of A's (also of C's and D's). These are again Deming's words, and echo Dennis Robert's comments about the pure subjectivity of the grading process. The motivation for the students should be in Joy of Learning (one of Deming's 14 points) rather than the grade. This I agree with wholeheartedly. How can we achieve this? I think it is our main challenge as educators. Using the grading system as a motivational substitute for Joy of Learning is lazy, inefficient management of our classes. Students who are fairly sure they are not going to get the coveted A, or who only need a "C or better" are going to give less effort. This will increase variation, and operates contrary to the stated goal of the system. My question is again: Is ranking really necessary? Given the goal of reducing variation, what does it help? Students in competition for the scarce A's will withhold information from one another. Does this achieve the stated aim of the system in an optimal way? W. Edwards Deming would have said, most emphatically, no. He spoke quite often of the educational system particularly in his later years; his message was not at all meant to be limited to manufacturing. Grading is not equivalent to ranking, unless one uses a forced distribution. One can grade without any restriction on the number of As or other grades other than the achievement of the students. I would be interested in hearing about any empirical evidence that non-use of grading schemes produces better or even as good learning as the use of grades? I think this is a very important point: what can we do in place of ranking? Now, as much as you say you don't use ranking, I am not sure you can get away without out. What if all of a sudden everyone got A's by your criteria? Wouldn't the administration get on your case? Then, you might say, just make the criteria harder so that we get back to a "normal" proportion of As, Bs etc. Well, aren't you just back to ranking? I don't have any data from the classroom experience, but I do have an observation from business. Texas Instruments had a policy of ranking plants in terms of their performance. The employees at the top plants received bonuses. Great idea, right? Motivates people, makes them perform to the best of their abilities, just like grading. The problem is, the innovations were hoarded by the individual plants to secure the bonuses, to the detriment of the company at large. Optimization of individual processes can be detrimental to the system, if the system at large is not considered in the optimization process. Thanks for the continuing discussion. I have been profoundly influenced by the words of W. Edwards Deming, and hope others will take a look at what he had to say, at least to stimulate discussions such as this. As he himself said, you don't simply "implement" his system, much like you don't learn to play piano by buying one and placing it in your living room. In the same way, you don't simply implement Deming's method as it applies to teaching by implementing P/F and be done with it. I would like to know, are there any others out there who have been influenced by Deming? Has his message lost its force in our current climate of economic prosperity? Peter Westfall
Re: adjusting marks
"David A. Heiser" wrote: - Original Message ----- From: Peter Westfall [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Tuesday, December 21, 1999 6:45 PM Subject: Re: adjusting marks Bob Hayden wrote: - Forwarded message from Peter Westfall - Deming himself (if I remember correctly) graded everyone as "A" until the administration noticed, and then they made his courses Pass-Fail. Deming was also very much against ranking students in any way, except for the possible exception of identifying an exceptional student that others might emulate (the 3*sigma student) and identifying the exceptionally poor student ( 3*sigma) for remediation. All other students should be be essentially equivalent, in Deming's philosophy. - End of forwarded message from Peter Westfall - Would you recommend this for drivers' license tests? Oh, I get it, that's what we're doing already! No wonder. I have to admit, it would sure simplify quality control if we considered anything within +- 3 s.d. to be OK. Then I guess the motivation would be to throw in a few clunkers now and then to keep the s.d. as large as possible? Bob, Your remarks sound facetious. I was hoping to stimulate some serious discussion. Have you read anything by Deming? Here is Deming's philosophy, as well as I can paraphrase it for the present situation: Students/teachers/administrators form a system. The system has an aim, which is (presumably) to educate everyone as well as possible, for the good of the students, and for the good of society. What good does ranking do? Does it help to achieve the aim of the system? Or rather, is it simply a weeding process? Is ranking necessary? (these are mainly Deming's words, but I must admit I see lots of value there.) Regarding making the standard deviation large, Deming would say that management's (professors, administrators) job entails minimizing variation among students. This can be done in the usual ways - admissions procedures, advising, prerequisites. Individual classes are "processes" within the larger system, and in the process of continual improvement, one seeks ways to minimize variation within the processes. Deming shows a diagram where the knowledge of people before training is scattered and highly variable, and after training the mean level is higher but the variation smaller. The inference is that the more effective the classroom experience, the less variation in the final levels of knowledge and abilities of the students, as they pertain to the subject at hand. My question is again: Is ranking really necessary? Given the goal of reducing variation, what does it help? Students in competition for the scarce A's will withhold information from one another. Does this achieve the stated aim of the system in an optimal way? W. Edwards Deming would have said, most emphatically, no. He spoke quite often of the educational system particularly in his later years; his message was not at all meant to be limited to manufacturing. Peter --- Very Intersting I don't agree with Demming. Life is essentially a matter of diversity, and being able to find one's own "niche". The process of ranking is inherent in life whenever there is stress on a population. Going to college is indeed "stress". If in order to suceed, I need to obtain a PhD from Stanford, then I have to get high grades and attain other acheivments to get in that few percent that gets accepted. If my college grades are all "pass", how am I going to compete with the applicate with A+++ grades from NCU? How are new hires for the expensive New York/Washington law firms hired? Not on pass/fail but on which law school and how the professors rated the student and what were the extra curricular activities? Much of this is subjective, but when you have 300 applicants for one job, you have to do some ranking to pick the top 3 or 5. Demming I think has the quality control mindset of pass/fail in terms of manufactured objects, where everything is acceptable between -3 and +3 sigma (Now it is -6 to +6 sigma.) This may be fine for shop work on the floor. In (I think Deming had some serious problems with 6 sigma QC, but that is besides the point.) this country the only thing we manufacture now is credit and money to buy manufactured goods from other countries. You need a very diverse population now. The process of ranking as flawed as it is, works, because there are so many areas where one can find his own niche, and ranking is one way of finding one's niche. DAH No doubt about it, we can't make everyone the same, nor do we want to. We can, however, make their levels of understanding and logical thought processes similar through proper educati
Re: adjusting marks
dennis roberts wrote: At 02:34 PM 12/21/99 -0600, EAKIN MARK E wrote: Dennis Roberts writes: i said this ... third ... usually, "curving" means lowering the cutoffs ... that were established at the beginning of a course (maybe in the syllabus) if that is the case ... then there is NO statistical rationale for this ... simply, your "gut" feeling that not enough students are making As, Bs, etc ... SO, you move the cutoffs down until YOU feel comfortable ... and mark countered In the case of my teaching philosoply, I will have to disagree with the above. To me, a student's grade can be expressed as but, i counter counter with ... sorry ... grading is PRIMARILY a subjective activity ... there is no other way to put it. now, you can have test scores, project scores, other observations, speeches, homework, knowledge from previous classes, etc. ... you name it. but, in the final analysis ... you put all this stuff together ... and then you DECIDE where to put the cut points ... and, if anyone out there thinks the placing of cut points in typical classes in schools is objective ... then merry christmas to you and to all a good night! == dennis roberts, penn state university educational psychology, 8148632401 http://roberts.ed.psu.edu/users/droberts/droberts.htm I agree with Dennis, and would like to chime in with some other points. The late W. Edwards Deming stated that the use of a forced distribution for grades is "ruinous" to the entire system of education. It seems to me that grading "on the curve" is in some sense an attempt at using a forced distribution. (And if the goal is indeed to enforce a distribution, then use the ranked data, not the normal distribution; see below for more about ranking.) Use of a forced distribution creates a win-lose scenario for the students. If we are to improve as educators, we need to seek win-win scenarios. Deming himself (if I remember correctly) graded everyone as "A" until the administration noticed, and then they made his courses Pass-Fail. Deming was also very much against ranking students in any way, except for the possible exception of identifying an exceptional student that others might emulate (the 3*sigma student) and identifying the exceptionally poor student ( 3*sigma) for remediation. All other students should be be essentially equivalent, in Deming's philosophy. I would be curious to hear what others have to say about this. Is Deming still with us? And how can we create win-win teaching strategies that will also satisfy administrators? Peter
Re: adjusting marks
Bob Hayden wrote: - Forwarded message from Peter Westfall - Deming himself (if I remember correctly) graded everyone as "A" until the administration noticed, and then they made his courses Pass-Fail. Deming was also very much against ranking students in any way, except for the possible exception of identifying an exceptional student that others might emulate (the 3*sigma student) and identifying the exceptionally poor student ( 3*sigma) for remediation. All other students should be be essentially equivalent, in Deming's philosophy. - End of forwarded message from Peter Westfall - Would you recommend this for drivers' license tests? Oh, I get it, that's what we're doing already! No wonder. I have to admit, it would sure simplify quality control if we considered anything within +- 3 s.d. to be OK. Then I guess the motivation would be to throw in a few clunkers now and then to keep the s.d. as large as possible? Bob, Your remarks sound facetious. I was hoping to stimulate some serious discussion. Have you read anything by Deming? Here is Deming's philosophy, as well as I can paraphrase it for the present situation: Students/teachers/administrators form a system. The system has an aim, which is (presumably) to educate everyone as well as possible, for the good of the students, and for the good of society. What good does ranking do? Does it help to achieve the aim of the system? Or rather, is it simply a weeding process? Is ranking necessary? (these are mainly Deming's words, but I must admit I see lots of value there.) Regarding making the standard deviation large, Deming would say that management's (professors, administrators) job entails minimizing variation among students. This can be done in the usual ways - admissions procedures, advising, prerequisites. Individual classes are "processes" within the larger system, and in the process of continual improvement, one seeks ways to minimize variation within the processes. Deming shows a diagram where the knowledge of people before training is scattered and highly variable, and after training the mean level is higher but the variation smaller. The inference is that the more effective the classroom experience, the less variation in the final levels of knowledge and abilities of the students, as they pertain to the subject at hand. My question is again: Is ranking really necessary? Given the goal of reducing variation, what does it help? Students in competition for the scarce A's will withhold information from one another. Does this achieve the stated aim of the system in an optimal way? W. Edwards Deming would have said, most emphatically, no. He spoke quite often of the educational system particularly in his later years; his message was not at all meant to be limited to manufacturing. Peter _ | | Robert W. Hayden | | Department of Mathematics / | Plymouth State College MSC#29 | | Plymouth, New Hampshire 03264 USA | * | Rural Route 1, Box 10 /| Ashland, NH 03217-9702 | ) (603) 968-9914 (home) L_/ [EMAIL PROTECTED] fax (603) 535-2943 (work)
