If you reply to this long (12 kB) post please don't hit the reply button unless you prune the copy of this post that may appear in your reply down to a few relevant lines, otherwise the entire already archived post may be needlessly resent to subscribers.
************************************************* ABSTRACT: In a recent popular TIPS thread "assessment question (AAAAUUUUGGGGHHHH)" psychologists commendably focus on testing so as to assess the effectiveness of their department's program for their *majors*. But how about the effectiveness of the GENERAL INTRODUCTORY PSYCHOLOGY COURSE given to tens of thousands of *both majors and nonmajors* nationwide every year ? Most psychologists appear to be either dismissive or oblivious of the fact that "Conceptual Inventories" <http://en.wikipedia.org/wiki/Concept_inventory>, developed through arduous quantitative and qualitative research by disciplinary experts, are currently being used to improve undergraduate - and some high-school - courses in science, technology, engineering, and math (STEM) disciplines (BUT NOT PSYCHOLOGY!) ************************************************* Marte Fallshore (2010) in his TIPS (Teaching In the Psychological Sciences) post of 25 Feb 2010 titled "RE: assessment question (AAAAUUUUGGGGHHHH)" initiated a 16 post (as of 2 March 2010 09:5:00-0800) TIPS thread by writing [bracketed by lines "FFFFFF. . . ."; slightly edited]: FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF . . . . My school, like all the others, is obsessed with assessment. . . . . I was wondering if anyone out there does a pre-posttest assessment of psych graduates? My chair is wanting to start something like that because we now have a 1-credit introduction to the major class when they declare. We want to give them the pretest in the majors class then a posttest during their senior assessment class. What do they know before the major and what do they know after? Anybody got any tests already written (and maybe normed) we could use? FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF To which Julie Osland (2010) responded: "One option would be require them to pay to take the ETS major field test in psychology for both classes." And Claudia Stanny (2010) responded: SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS If you are focused entirely on content and fact retrieval, a pre-post test doesn't pose a very interesting question. You could probably answer it better by using something like the Major Fields test for psychology (ETS) and then look at subtest scores to look at knowledge areas to identify areas of strengths and weaknesses. I'm assuming ETS provides these subscores for areas in psychology for the Psychology test. . . . . . If you would really like to have some sort of baseline for content knowledge, you could volunteer to participate in the College Board development of norms for the AP Psychology exam. Students take the AP exam at the end of their introductory psychology course. Not exactly entering the major, but I hope they learn more about the content of psychology in all those other courses they take later! It would be sad if they learned all the relevant content in intro! :-) SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS In my opinion, the above focus on testing so as to assess the effectiveness of a psychology department's program for their *majors* is commendable (even despite Fallshore's emotive "AAAAUUUUGGGGHHHH") but how about the effectiveness of the GENERAL INTRODUCTORY PSYCHOLOGY COURSE given to tens of thousands of *both majors and nonmajors* nationwide every year ? In a recent post [Hake (2010a)] regarding the book "Quality Research in Literacy and Science Education: International Perspectives and Gold Standards" [Shelley et al. (2009)] I criticized one aspect of the contribution of Robin Millar (Salters' Professor of Science Education at the University of York, UK) and Jonathan Osborne (holder of the Endowed Chair of Science Education at Stanford University) to that book, but the same comment would apply to psychologists generally. I wrote: HHHHHHHHHHHHHHHHHHHHHHHHHHHH Millar and Osborne appear to be either dismissive or oblivious of the fact that "Conceptual Inventories" <http://en.wikipedia.org/wiki/Concept_inventory>, developed through arduous quantitative and qualitative research by disciplinary experts, are currently being used to improve undergraduate - and some high-school - courses in science, technology, engineering, and math (STEM) disciplines (BUT NOT PSYCHOLOGY!) - see, e.g., (a) "Design-Based Research in Physics Education Research: A Review" (Hake (2008); (b) "Should We Measure Change? Yes!" (Hake, 2010b); and (c) "Workshop on Linking Evidence and Promising Practices in STEM Undergraduate Education" [National Academies (2008)]. . . . [[and - added on 2 March 2010 - "Do Psychologists Research the Effectiveness of Their Courses? Hake Responds to Sternberg" (Hake, 2005)]]. . . . . HHHHHHHHHHHHHHHHHHHHHHHHHHHH Richard Hake, Emeritus Professor of Physics, Indiana University 24245 Hatteras Street, Woodland Hills, CA 91367 Honorary Member, Curmudgeon Lodge of Deventer, The Netherlands <rrh...@earthlink.net> <http://www.physics.indiana.edu/~hake/> <http://www.physics.indiana.edu/~sdi/> <http://HakesEdStuff.blogspot.com/> <http://iub.academia.edu/RichardHake> "What we assess is what we value. We get what we assess, and if we don't assess it, we won't get it." Lauren Resnick [quoted by Grant Wiggins (1990)] REFERENCES [Tiny URL's courtesy <http://tinyurl.com/create.php>.] Fallshore, M. 2010. "assessment question (AAAAUUUUGGGGHHHH)"; TIPS post of 25 Feb 2010 12:21:41-0800; online on the OPEN! TIPS archives at <http://www.mail-archive.com/tips%40fsulist.frostburg.edu/msg00392.html>. Hake, R.R. 2005. "Do Psychologists Research the Effectiveness of Their Courses? Hake Responds to Sternberg," online on the OPEN! AERA-L archives at <http://tinyurl.com/yll6r2n>. Post of 21 Jul 2005 22:55:31-0700 to AERA-C, AERA-D, AERA-J, AERA-L, ASSESS, EvalTalk, POD, PhysLrnR, and STLHE-L. Hake, R.R. 2006. "Possible Palliatives for the Paralyzing Pre/Post Paranoia that Plagues Some PEP's" [PEP's = Psychologists, Education Specialists, and Psychometricians], Journal of MultiDisciplinary Evaluation, Number 6, November, online at <http://survey.ate.wmich.edu/jmde/index.php/jmde_1/article/view/41/50>. This even despite the admirable anti-alliteration advice at psychologist Donald Zimmerman's site <http://mypage.direct.ca/z/zimmerma/> to "Always assiduously and attentively avoid awful, awkward, atrocious, appalling, artificial, affected alliteration." Hake, R.R. 2008. "Design-Based Research in Physics Education Research: A Review," in Kelly, Lesh, & Baek (2008)]. A pre-publication version of that chapter is online at <http://www.physics.indiana.edu/~hake/DBR-Physics3.pdf> (1.1 MB). Hake, R.R. 2010a. "Re: Quality Research in Literacy and Science Education: International Perspectives and Gold Standards," online on the OPEN! AERA-L archives at <http://tinyurl.com/yhhbu72>. Post of 22 Feb 2010 14:04:43-0800 to AERA-L and Net-Gold. The abstract was sent to various discussion list and also appears at <http://hakesedstuff.blogspot.com/2010/02/re-quality-research-in-literacy-and.html> with a provision for comments. Hake, R.R. 2010b. "Should We Measure Change? Yes!" online at <http://www.physics.indiana.edu/~hake/MeasChangeS.pdf> (2.5 MB) and as ref. 43 at <http://www.physics.indiana.edu/~hake>. To appear as a chapter in "Evaluation of Teaching and Student Learning in Higher Education" [Hake (in preparation)]. For a severely truncated version see Hake (2006). Kelly, A.E., R.A. Lesh, & J.Y. Baek. 2008. "Handbook of Design Research Methods in Education: Innovations in Science, Technology, Engineering, and Mathematics Learning and Teaching." Routledge. Publisher's information at <http://tinyurl.com/4eazqs>; Amazon.com information at <http://tinyurl.com/5n4vvo>. National Academies. 2008. "Workshop on Linking Evidence and Promising Practices in STEM Undergraduate Education": Commissioned Papers at <http://www7.nationalacademies.org/bose/PP_Commissioned_Papers.html>. Osland, J. 2010. RE: assessment question (AAAAUUUUGGGGHHHH)"; TIPS post of 25 Feb 2010 12:00:06-0800; online on the OPEN! TIPS archives at <http://www.mail-archive.com/tips%40fsulist.frostburg.edu/msg00389.html>. Shelley, M.C., L.D. Yore, & B. Hand, eds. 2009. "Quality Research in Literacy and Science Education: International Perspectives and Gold Standards." Springer, publisher's information at <http://www.springerlink.com/content/g2447682464446x2/>. Amazon.com information at <http://tinyurl.com/yf7efra>, note the searchable "Look Inside" feature. Barnes & Noble information at <http://tinyurl.com/y8n9pe9>. An expurgated (teaser) version is online as a Google "book preview" at <http://tinyurl.com/yddphh3>. Stanny, C. 2010. RE: assessment question (AAAAUUUUGGGGHHHH)"; TIPS post of 25 Feb 2010 12:08:58-0800; online on the OPEN! TIPS archives at <http://www.mail-archive.com/tips%40fsulist.frostburg.edu/msg00391.html>. Wiggins, G. 1990. "The Truth May Make You Free, But the Test May Keep You Imprisoned: Toward Assessment Worthy of the Liberal Arts," AAHE Assessment Forum: 17-31; online <http://www.maa.org/saum/articles/wiggins_appendix.html>. --- You are currently subscribed to tips as: arch...@jab.org. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13090.68da6e6e5325aa33287ff385b70df5d5&n=T&l=tips&o=1006 or send a blank email to leave-1006-13090.68da6e6e5325aa33287ff385b70df...@fsulist.frostburg.edu
