The American TV news programme 60 Minutes had an interesting segment tonight on the Harlem's Children's Zone (see video at http://www.cbsnews.com/sections/60minutes/main3415.shtml )
It concerns an attempt to close the racial achievement gap between Black and White by a massive application of free social, medical, and educational services to 10,000 children living in one area of Harlem. The results are said to be "stunning". But there is reason for caution. Not all well-intentioned programmes have the effects anticipated for them. The cardinal example is the Cambridge-Somerville Youth Study of the 1930's which had similar aims. Because of the care with which that early study had been constructed, including a randomized control group, it was possible to go back many years later to assess its outcomes. Surprisingly and disturbingly, they were negative: the only demonstrated effects of the programme were detrimental to its participants (see, for example http://www.bsos.umd.edu/gvpt/lpbr/subpages/reviews/mccord11 07.htm or http://tinyurl.com/ycavz4m ). So one can ask how well the evaluation of the new Children's Zone project was carried out and how well-supported its claim of stunning success is. One detail intrigued me. For some of their cohorts, more parents wanted their children to enrol than the project could handle, so admission was by lottery. This converts this project into a randomized experiment, comparing lottery winners (project participants) with lottery losers (wait-list participants). With a bit of searching I found a document reporting outcomes dated April 2009 but otherwise unidentified and probably therefore unpublished. It's: Are High-Quality Schools Enough to Close the Achievement Gap? Evidence from a Bold Social Experiment in Harlem Will Dobbie and Roland G. Fryer, Jr. Harvard University http://www.economics.harvard.edu/faculty/fryer/files/hcz%204.1 5.2009.pdf or http://tinyurl.com/qh8npt Unfortunately, the document is a massive tome and not readily accessible to a casual read. But one critical set of figures, which appears to support the claim of stunning success, puzzles me. The figures are on p. 32 and 33 (Figures 3A and 3B). They show math and ELA (presumably English Language Arts) test results as a function of grade and lottery status. They appear to show that the achievement gap between White and Black students has been erased for math, although not for ELA. But a footnote warns that the lottery winners are not what they seem. They are defined as " students who receive a winning lottery number or who are in the top ten of the waitlist." The text on p. 14 provides further information: "Lottery winners are comprised of students who either won the lottery, were in the top ten of individuals on the wait list, or who had a sibling that is already enrolled in the Promise Academy. Lottery losers are individuals who lost the lottery and were eleven or below on the waiting list". I don't understand what they mean by "top ten of individuals on the wait list", and I hope it's not a ranking by academic qualification to enter the programme. More likely, these are the students who just happen to be first on the list to be called if a vacancy opens up. That would mean that the lottery winner group consists of lottery winners (and siblings already in the programme?), together with an (unspecified?) number of students who either never entered the programme, or who entered it for a shorter period of time. If so, the results would likely be stronger if these cases were not included, because the experimental group would be diluted by cases which did not receive the full treatment. Why would they conduct their analysis in this way? Why not restrict the lottery winners group to those who just won the lottery, full stop? Can anyone help me out here? Stephen ----------------------------------------------------------------- Stephen L. Black, Ph.D. Professor of Psychology, Emeritus Bishop's University e-mail: sbl...@ubishops.ca 2600 College St. Sherbrooke QC J1M 1Z7 Canada ----------------------------------------------------------------------- --- To make changes to your subscription contact: Bill Southerly (bsouthe...@frostburg.edu)
