I don't think the replication results are very surprising. It seems
to be closely related to the "file drawer problem" and publication policy.
Suzy Q and a bunch of others are, independently, studying the same effect. The
effect happens to be pretty small, and the sample sizes Suzy and others use not
very large. Sampling error leads half of the researchers to underestimate the
magnitude of the effect, half overestimating it. Suzy is the lucky one who
gets data leading to an overestimate sufficiently large to produce significant
results. Now this is not a Type I error, there is an effect, but she has,
unknowingly, overestimated the magnitude of the effect. It should not be
surprising that when others try to replicate it, they get much smaller
estimates of the magnitude of the effect.
Yes, meta-analysis is called for here, if you can get the data out of
all those file drawers.
Cheers,
[Karl L. Wuensch]<http://core.ecu.edu/psyc/wuenschk/klw.htm>
From: Jim Clark [mailto:[email protected]]
Sent: Friday, August 28, 2015 3:41 PM
To: Teaching in the Psychological Sciences (TIPS)
Subject: Re: [tips] PSYC failed the replication test
Hi
Some notable lapses I think. No mention of meta-analysis that I saw. And the
Science editor's comment that marginal results not be published is exactly
wrong. Every finding should be published so that studies can be aggregated.
Psychology made a big mistake when it turned from small papers consisting
mostly of methods and results for single studies to mega papers with multiple
studies and (inflated?) intros and discussions. Thes problems are exacerbated
by calls for larger and larger Ns.
Jim
Sent from my iPhone
On Aug 28, 2015, at 2:33 PM, "Dennis Goff"
<[email protected]<mailto:[email protected]>> wrote:
You all might find an article published on the Five Thirty Eight website (link
http://fivethirtyeight.com) useful when talking about this issue with
students. The article includes a very nice interactive demonstration of
p-hacking that I think will help students to understand that concept. Beyond
that, the article takes a balanced approach to the problems of replicability,
the growth in the number of journal articles being published, fraud and other
related topics. The title is Science Isn't Broken by , Christie Aschwanden.
Here is the link http://fivethirtyeight.com/features/science-isnt-broken/
Dennis
----------------------------------------------------------
Dennis M Goff
Charles A Dana Professor of Psychology
Chair Department of Psychology
Office: (434) 947-8547
Randolph College
Founded as Randolph-Macon Woman's College in 1891
2500 Rivermont Avenue
Lynchburg, VA 24503
www.randolphcollege.edu<http://www.randolphcollege.edu/>
From: Rick Froman [mailto:[email protected]]
Sent: Friday, August 28, 2015 2:17 PM
To: Teaching in the Psychological Sciences (TIPS)
<[email protected]<mailto:[email protected]>>
Subject: RE:[tips] PSYC failed the replication test
It is true that we only got a D+ :( but 68% is quite a bit higher :) than some
of the estimates of reproducibility I have heard bandied about (which are close
to zero).
Rick
Dr. Rick Froman<http://bit.ly/16z4vcd>
Professor of Psychology
Box 3519
John Brown University
2000 W. University Siloam Springs, AR 72761
[email protected]<mailto:[email protected]>
(479) 524-7295
From: Wuensch, Karl Louis [mailto:[email protected]]
Sent: Friday, August 28, 2015 12:59 PM
To: Teaching in the Psychological Sciences (TIPS)
<[email protected]<mailto:[email protected]>>
Subject: [tips] PSYC failed the replication test
This article is going on the reading list for my grad students in
stats. Once we have covered power and publication biases this article should,
I hope, lead to some lively discussion.
http://www.sciencemag.org/content/349/6251/aac4716
Cheers,
________________________________
<image001.jpg><http://www.ecu.edu/>
Karl L. Wuensch, Professor and ECU Scholar/Teacher, Dept. of Psychology
East Carolina University, Greenville NC 27858-4353, USA,
Earth<http://core.ecu.edu/psyc/wuenschk/Earth.htm>
Associate Editor, Society & Animals
Voice: 252-328-9420 Fax: 252-328-6283
http://core.ecu.edu/psyc/wuenschk/klw.htm
---
You are currently subscribed to tips as:
[email protected]<mailto:[email protected]>.
To unsubscribe click here:
http://fsulist.frostburg.edu/u?id=13039.37a56d458b5e856d05bcfb3322db5f8a&n=T&l=tips&o=46469
(It may be necessary to cut and paste the above URL if the line is broken)
or send a blank email to
leave-46469-13039.37a56d458b5e856d05bcfb3322db5...@fsulist.frostburg.edu<mailto:leave-46469-13039.37a56d458b5e856d05bcfb3322db5...@fsulist.frostburg.edu>
---
You are currently subscribed to tips as:
[email protected]<mailto:[email protected]>.
To unsubscribe click here:
http://fsulist.frostburg.edu/u?id=13169.2bfd597f06c032f81efb35e857e2dd91&n=T&l=tips&o=46470
(It may be necessary to cut and paste the above URL if the line is broken)
or send a blank email to
leave-46470-13169.2bfd597f06c032f81efb35e857e2d...@fsulist.frostburg.edu<mailto:leave-46470-13169.2bfd597f06c032f81efb35e857e2d...@fsulist.frostburg.edu>
---
You are currently subscribed to tips as:
[email protected]<mailto:[email protected]>.
To unsubscribe click here:
http://fsulist.frostburg.edu/u?id=13251.645f86b5cec4da0a56ffea7a891720c9&n=T&l=tips&o=46472
(It may be necessary to cut and paste the above URL if the line is broken)
or send a blank email to
leave-46472-13251.645f86b5cec4da0a56ffea7a89172...@fsulist.frostburg.edu<mailto:leave-46472-13251.645f86b5cec4da0a56ffea7a89172...@fsulist.frostburg.edu>
---
You are currently subscribed to tips as:
[email protected]<mailto:[email protected]>.
To unsubscribe click here:
http://fsulist.frostburg.edu/u?id=13060.c78b93d4d09ef6235e9d494b3534420e&n=T&l=tips&o=46473
(It may be necessary to cut and paste the above URL if the line is broken)
or send a blank email to
leave-46473-13060.c78b93d4d09ef6235e9d494b35344...@fsulist.frostburg.edu<mailto:leave-46473-13060.c78b93d4d09ef6235e9d494b35344...@fsulist.frostburg.edu>
---
You are currently subscribed to tips as: [email protected].
To unsubscribe click here:
http://fsulist.frostburg.edu/u?id=13090.68da6e6e5325aa33287ff385b70df5d5&n=T&l=tips&o=46481
or send a blank email to
leave-46481-13090.68da6e6e5325aa33287ff385b70df...@fsulist.frostburg.edu
