I distinguish between a context of discovery, where one SHOULD massage data to discover things (serendipity), and a context of justification (publication) in which we try to convince others of our conclusions. If a researcher (using more than just p = such and such) really believes they have found something, they should try to publish it. It's the responsibility of the reviewers and editors to judge whether the conclusions are warranted, hopefully also using more than p = etc. The best hedge against Type I errors is replication, and getting it published is a way to invite replication. So I'd say kosher .. or at least neutral!
-------------------------- John W. Kulig Professor of Psychology Plymouth State University Plymouth NH 03264 -------------------------- ----- Original Message ----- From: "Don Allen" <dal...@langara.bc.ca> To: "Teaching in the Psychological Sciences (TIPS)" <tips@acsun.frostburg.edu> Sent: Monday, December 7, 2009 11:44:24 AM GMT -05:00 US/Canada Eastern Subject: Re: [tips] Birth order effects for cooperation? Hi Stephen- Looks like a case of data-mining to me as well. Unless they show an apriori rationale for such a strange grouping then I would disregard their findings. -Don. ----- Original Message ----- From: sbl...@ubishops.ca Date: Monday, December 7, 2009 8:11 am Subject: [tips] Birth order effects for cooperation? To: "Teaching in the Psychological Sciences (TIPS)" > OK, here's another study I'm mulling over. Courtiol et al (2009) > have just reported an experiment on cooperation in college > students as a function of birth order. Their measure of > cooperation is an objective one, taken from the results of a two- > person game. The game provides numerical values for trust and > reciprocity, determined by how much money each player sends > or returns to his partner. Although birth order studies are > infested with methodological problems, this design, as far as I > can see, successfully avoids them. > > The history of claims for birth order effects is not a happy one > (e.g. see Judith Rich Harris' "Four Essays on Birth Order" > (2004) at http://xchar.home.att.net/tna/birth-order/index.htm > and > also her more recent review in "No Two Alike" (2006)--the > chapter headed 'Birth Order and Other Environmental > Differences Within the Family"). So I paid attention when > Courtiol et al reported positive effects of birth order on both > trust > and reciprocity. > > But here's the catch. They provided a complex statistical > analysis (to me, anyway) but their analysis depends on a > curious grouping of birth order: first-borns comprised one > group, > and later-borns the other. But the later-born group also > included > only children (without siblings). On logical grounds, one would > think that only children belong in the first-born category > instead. > > Their justification for doing this was inspection of the data. > For > trust: "Means of x [their monetary datum] for middleborn, > lastborn and only children appeared much closer to each other > than to the mean of x for firstborns (Table 2); these three > categories were therefore pooled." For reciprocity: "Only > children and laterborns were pooled because their average > amounts sent (y) were closer to each other than to the average > amount sent by firstborns (Table 2)." > > My own inspection of their data suggests that without this post- > hoc categorization, they would not have been able to report > significant results. Is their move kosher, or do we have a case > of > data-massaging here? > > _Science_ has a news item on the study at > http://tinyurl.com/ylc4l34 > It does not mention the peculiar definition of "later-borns". > > Stephen > > Courtiol, A. Raymond, M. and Faurie, C. (2009). Birth order > affects behaviour in the investment game: Firstborns are less > trustful and reciprocate less. Animal Behaviour, 78, 1405-1411. > > ----------------------------------------------------------------- > Stephen L. Black, Ph.D. > Professor of Psychology, Emeritus > Bishop's University > e-mail: sbl...@ubishops.ca > 2600 College St. > Sherbrooke QC J1M 1Z7 > Canada > ----------------------------------------------------------------- > ------ > > --- > To make changes to your subscription contact: > > Bill Southerly (bsouthe...@frostburg.edu) > Don Allen, Retired Formerly with: Dept. of Psychology Langara College 100 W. 49th Ave. Vancouver, B.C. Canada V5Y 2Z6 Phone: 604-733-0039 --- To make changes to your subscription contact: Bill Southerly (bsouthe...@frostburg.edu) --- To make changes to your subscription contact: Bill Southerly (bsouthe...@frostburg.edu)
