Hello Ken,
I'm teaching Psyc of Music again here at Lawrence, and I'd like to use the recent Schellenberg article and the critique thereof as exemplars of how to do (and not to do) research on questions about music psychology. I have a sketchy version of your critique from emails on the TIPS list, but you stated last August that you're seeking publication, possibly thru Am Psychologist. I haven't yet found that, so I was wondering whether you have indeed found an outlet for your critique. If not, would you be willing to send me a prepublication version of this to have my students read?
Thanks,
Terry Rew-Gottfried Lawrence University
Ken Steele wrote:
Christopher D. Green wrote:
Ronald C. Blue wrote:
http://www.psychologicalscience.org/media/releases/2004/pr040819.cfm
First Solid Evidence that the Study of Music Promotes Intellectual Development
There is some fairly tricky use of statistics in this study (which I think I would have questioned had I been a reviewer). See the table below (copied from the aticle). Note that the raw IQ increases (over 36 weeks) were 6.1 for keyboard, 7.4 for voice, 5.2 for drama and 3.9 for no lessons.
TABLE 1
Mean Sample Characteristics of the Four Groups of Children
Sample characteristic
Group
Keyboard Voice Drama No lessons
n before lessons 36 36 36 36
n after lessons 30 32 34 36
Age (days over 6 years) 74 (78) 102 (78) 75 (85) 113 (80)
Family income 4.6 (1.3) 4.3 (1.4) 4.3 (1.4) 4.1 (1.4)
Full-scale IQ 102.6 (8.8) 103.8 (10.9) 102.6 (13.6) 99.4 (9.7)
before lessons
Full-scale IQ 108.7 (12.5) 111.4 (12.6) 107.7 (13.8) 103.3 (9.9)
after lessons
-------------
Note. Standard deviations are in parentheses. Age is listed as days over 6 years on September 1 (before lessons). Family income was measured in increments of $25,000 (15less than $25,000; 25$25,000–$49,999; 35$50,000–$74,999; and so on). There was no difference among groups in age, family income, or full-scale IQ when the children were tested initially (before lessons). The difference among groups in full-scale IQ after the lessons was reliable (p5.05).
-------------
Fig. 1. Mean increase in full-scale IQ (Wechsler Intelligence Scale for Children–Third Edition) for each group of 6-year-olds who completed the study.
The analysis then proceeds as follows:
"The two music groups had increases of similar magnitude despite differences
in teaching style (p>.8). The two control groups also had similar increases (p>.7), which indicates that drama instruction is not an extracurricular activity associated with notable increases in intellectual development.
"Similarities between the two music groups and the two control groups justified collapsing the data across groups in order to maximize power in tests of the central hypothesis. Compared with the control groups, the music groups had reliably larger increases in full-scale IQ, t(130)51.99, p < .05."
First off, I do not understand what the p-values in the first sentence refer to -- "magnitudes" are not typically given as probabilities. Whatever they are, I'm not convinced that the "difference" of .8 and .7 "indicates" anything as definitively as the authors suggest, nor that it justifies grouping the different conditions in this (and only this) way. Note that the raw differences in IQ increases between keyboard and drama were LESS than those between keyboard and voice (so, why not, e.g., group keyboard and drama together, in which case the conclusion would be that voice lessons increase IQ, but not keyboard or drama?). Second, although it is said the conditions were grouped in order to "maximize power," it seems likely (to me, anyway) that they were grouped this way primarily because the obvious (and unreported) 1-way ANOVA that would have normally have been applied to this data came up non-significant and so another aproach had to be developed ad hoc.
Correct. And neither is the ANCOVA, using pretest IQ as the covariate.
Now none of this is out and out "wrong," it's just all a little "stretched" and, in any case, hardly justifies the "incontrovertable" label that APS gives the results in its PR blurb. I remain skeptical.
"Stretched" is charitable. I have done further analyses using the original data set, as provided to me by Shellenberg.
First, the combined "music lessons" obscures the fact that at no time are keyboard lessons signficantly different from drama, no lessons, or their combination. Only voice training is different.
Second, note that there was subject attrition in the voice condition. The mean IQ of the subjects that dropped out was lower that the mean IQ of the group. This means that the pottest mean IQ of the voice group was probably increased by subject attrition. Subject attrition is an alternative explanation of the effect for voice training.
I made these two points (and others) in a commentary that I submitted to Psych. Science. The Editor declined to publish the commentary but did compliment me on reading the article closely.
Ken
PS - I am shopping for an alternative outlet. A friend suggests American Psychologist :-)
Regards, -- Christopher D. Green Department of Psychology York University Toronto, Ontario, Canada M3J 1P3
e-mail: [EMAIL PROTECTED] phone: 416-736-5115 ext. 66164 fax: 416-736-5814 http://www.yorku.ca/christo/ ============================
--------------------------------------------------------------- Kenneth M. Steele, Ph.D. [EMAIL PROTECTED] Department of Psychology http://www.psych.appstate.edu Appalachian State University Boone, NC 28608 USA ---------------------------------------------------------------
---
You are currently subscribed to tips as: [EMAIL PROTECTED]
To unsubscribe send a blank email to [EMAIL PROTECTED]
--- You are currently subscribed to tips as: [email protected] To unsubscribe send a blank email to [EMAIL PROTECTED]
